mirror of
https://github.com/bellingcat/RS4OSINT.git
synced 2026-06-10 04:28:36 +03:00
eoghan edits + changed dir structure
This commit is contained in:
Ollie Ballinger
1
chapters/.gitignore
vendored
Normal file
1
chapters/.gitignore
vendored
Normal file
@@ -0,0 +1 @@
|
||||
/.quarto/
|
||||
@@ -1,69 +1,102 @@
|
||||
# Remote Sensing {.unnumbered}
|
||||
---
|
||||
title: Remote Sensing
|
||||
---
|
||||
|
||||
Before learning how to load, process, and analyze satellite imagery in Google Earth Engine, it will be helpful to know a few basic principles of remote sensing. This section provides a brief overview of some important concepts and terminology that will be used throughout the course, including active and passive sensors; spatial, spectral and temporal resolution; and orbits.
|
||||
|
||||
Before learning how to load, process, and analyze satellite imagery in Google Earth Engine, it will be helpful to know a few basic principles of remote sensing. This section provides a brief overview of some important concepts and terminology that will be used throughout the course, including active and passive sensors; spatial, spectral, and temporal resolution; and orbits.
|
||||
|
||||
## Active and Passive Sensors
|
||||
|
||||
[Remote sensing](https://www.sciencedirect.com/topics/medicine-and-dentistry/remote-sensing) is the science of obtaining information about an object or phenomenon without making physical contact with the object. Remote sensing can be done with various types of electromagnetic radiation such as visible, infrared, or microwave. The electromagnetic radiation is either emitted or reflected from the object being sensed. The reflected radiation is then collected by a sensor and processed to obtain information about the object.
|
||||
|
||||
|
||||
[Remote sensing](https://www.sciencedirect.com/topics/medicine-and-dentistry/remote-sensing) is the science of obtaining information about an object or phenomenon without making physical contact with the object. Remote sensing can be done with various types of electromagnetic radiation such as visible, infrared or microwave. The electromagnetic radiation is either emitted or reflected from the object being sensed. The reflected radiation is then collected by a sensor and processed to obtain information about the object.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
While most satellite imagery is optical, meaning it captures sunlight reflected by the earth’s surface, Synthetic Aperture Radar (SAR) satellites such as Sentinel-1 work by emitting pulses of radio waves and measuring how much of the signal is reflected back. This is similar to the way a bat uses sonar to “see” in the dark: by emitting calls and listening to echoes.
|
||||
|
||||
|
||||
## Resolution
|
||||
|
||||
|
||||
Resolution is one of the most important attributes of satellite imagery. There are three types of resolution: spatial, spectral, and temporal. Let's look at each of these.
|
||||
|
||||
|
||||
### Spatial Resolution
|
||||
|
||||
Spatial resolution governs how "sharp" an image looks. The Google Maps satellite basemap, for example, is really sharp
|
||||
|
||||
Spatial resolution governs how "sharp" an image looks. The Google Maps satellite basemap, for example, is really sharp.
|
||||
Most of the optical imagery that is freely available has relatively low spatial resolution (it looks more grainy than, for example, the Google satellite basemap),
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Spectral Resolution
|
||||
|
||||
|
||||
What open access imagery lacks in spatial resolution it often makes up for with *spectral* resolution. Really sharp imagery from MAXAR, for example, mostly collects light in the visible light spectrum, which is what our eyes can see. But there are other parts of the electromagnetic spectrum that we can't see, but which can be very useful for distinguishing between different materials. Many satellites that have a lower spatial resolution than MAXAR, such as Landsat and Sentinel-2, collect data in a wider range of the electromagnetic spectrum.
|
||||
|
||||
|
||||
Different materials reflect light differently. An apple absorbs shorter wavelengths (e.g. blue and green), and reflects longer wavelengths (red). Our eyes use that information-- the color-- to distinguish between different objects. Below is a plot of the spectral profiles of different materials:
|
||||
|
||||
|
||||
<iframe title="Spectral Profiles of Different Materials" aria-label="Interactive line chart" id="datawrapper-chart-b1kcX" src="https://datawrapper.dwcdn.net/b1kcX/3/" scrolling="no" frameborder="0" style="width: 0; min-width: 100% !important; border: none;" height="400"></iframe><script type="text/javascript">!function(){"use strict";window.addEventListener("message",(function(e){if(void 0!==e.data["datawrapper-height"]){var t=document.querySelectorAll("iframe");for(var a in e.data["datawrapper-height"])for(var r=0;r<t.length;r++){if(t[r].contentWindow===e.source)t[r].style.height=e.data["datawrapper-height"][a]+"px"}}}))}();
|
||||
</script>
|
||||
|
||||
The visible portion of the spectrum is highlighted on the left, ranging from 400nm (violet) to 700nm (red). Our eyes (and satellite imagery in the visible light spectrum) can only see this portion of the light spectrum; we can't see UV or infrared wavelengths, for example, though the extent to which different materials reflect or absorb these wavelengths is just as useful for distinguishing between them. The European Space Agency's Sentinel-2 satellite collects spectral information well beyond the visible light spectrum, enabling this sort of analysis. It chops the electromagnetic spectrum up into "bands", and measures how strongly wavelengths in each of those bands is reflected:
|
||||
|
||||
The visible portion of the spectrum is highlighted on the left, ranging from 400 nanometers (violet) to 700nm (red). Our eyes (and satellite imagery in the visible light spectrum) can only see this portion of the light spectrum; we can't see UV or infrared wavelengths, for example, though the extent to which different materials reflect or absorb these wavelengths is just as useful for distinguishing between them. The European Space Agency's Sentinel-2 satellite collects spectral information well beyond the visible light spectrum, enabling this sort of analysis. It chops the electromagnetic spectrum up into "bands", and measures how strongly wavelengths in each of those bands is reflected:
|
||||
|
||||
|
||||
|
||||
|
||||
To illustrate why this is important, consider Astroturf (fake plastic grass). Astroturf and real grass will both look green to us, espeically from a satellite image. But living plants strongly reflect radiation from the sun in a part of the light spectrum that we can't see (near-infrared). There's a spectral index called the Normalized Difference Vegetation Index (NDVI) which exploits this fact to isolate vegetation in multispectral satellite imagery. So if we look at [Gilette Stadium](https://en.wikipedia.org/wiki/Gillette_Stadium) near Boston, we can tell that the three training fields south of the stadium are real grass (they generate high NDVI values, showing up red), while the pitch in the stadium itself is astroturf (generating low NDVI values, showing up blue).
|
||||
|
||||
|
||||
To illustrate why this is important, consider Astroturf (fake plastic grass). Astroturf and real grass will both look green to us, especially from a satellite image. But living plants strongly reflect radiation from the sun in a part of the light spectrum that we can't see (near-infrared). There's a spectral index called the Normalized Difference Vegetation Index (NDVI) which exploits this fact to isolate vegetation in multispectral satellite imagery. So if we look at [Gillette Stadium](https://en.wikipedia.org/wiki/Gillette_Stadium) near Boston, we can tell that the three training fields south of the stadium are real grass (they generate high NDVI values, showing up red), while the pitch in the stadium itself is astroturf (generating low NDVI values, showing up blue).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
In other words, even though these fields are all green and indistinguishable to the human eye, their *spectral profiles* beyond the visible light spectrum differ, and we can use this information to distinguish between them.
|
||||
|
||||
Astroturf is a trivial example. But suppose we were interested in identifying makeshift oil refineries in Northern Syria that constitute a key source of rents for whichever group controls them. As demonstrated in the ['Refinery Identification'](refineries.qmd) case study, we can train an algorithm to identify the spectral signatures of oil, and use that to automatically detect them in satellite imagery.
|
||||
|
||||
Astroturf is a trivial example. But suppose we were interested in identifying makeshift oil refineries in Northern Syria that constitute a key source of rents for whichever group controls them. As demonstrated in the ['Refinery Identification'](C2_Refineries.qmd) case study, we can train an algorithm to identify the spectral signatures of oil, and use that to automatically detect them in satellite imagery.
|
||||
|
||||
|
||||
|
||||
|
||||
### Temporal Resolution
|
||||
|
||||
|
||||
Finally, the frequency with which we can access new imagery is an important consideration. This is called the **temporal resolution**.
|
||||
|
||||
|
||||
The Google Maps basemap is very high resolution, available globally, and is freely available. But it has no *temporal* dimension: it's a snapshot from one particular point in time. If the thing we're interested in involves *changes* over time, this basemap will be of limited use.
|
||||
|
||||
|
||||
The **"revisit rate"** is the amount of time it takes for the satellite to pass over the same location twice. For example, the Sentinel-2 constellation's two satellites can achieve a revisit rate of 5 days, as shown in this cool video from the European Space Agency:
|
||||
|
||||
|
||||
{{< video https://dlmultimedia.esa.int/download/public/videos/2016/08/004/1608_004_AR_EN.mp4 >}}
|
||||
|
||||
Some satellite constellations are able to achieve much higher revisit rates. Sentinel-2 has a revisit rate of 5 days, but SkySat capable of imaging the same point on earth around 12 times per day! How is that possible? Well, as the video above demonstrated, the Sentinel-2 constellation is composed of two satellites that share the same orbit, 180 degrees apart. In contrast, the SkySat constellation comprises 21 satellites, each with its own orbital path:
|
||||
|
||||
Some satellite constellations are able to achieve much higher revisit rates. Sentinel-2 has a revisit rate of 5 days, but SkySat is capable of imaging the same point on earth around 12 times per day! How is that possible? Well, as the video above demonstrated, the Sentinel-2 constellation is composed of two satellites that share the same orbit, 180 degrees apart. In contrast, the SkySat constellation comprises 21 satellites, each with its own orbital path:
|
||||
|
||||
{{< video https://assets.planet.com/products/hi-res/Planet_Block_3_HD_1080p.mp4 >}}
|
||||
|
||||
|
||||
This allows SkySat to achieve a revisit rate of 2-3 *hours*. The catch, however, is that you need to pay for it (and it [ain't cheap](https://apollomapping.com/blog/an-update-on-skysat-tasking-pricing-and-video-capabilities)). Below is a comparison of revisit rates for various other optical satellites:
|
||||
|
||||
|
||||
](images/revisit_chart.png)
|
||||
|
||||
|
||||
## Summary
|
||||
|
||||
You should hopefully have a better understanding of what satellite imagery is, and how it can be used to answer questions about the world. In the [next section](ch2.qmd), we'll look at the various types of satellite imagery stored in the Google Earth Engine catalogue.
|
||||
|
||||
You should hopefully have a better understanding of what satellite imagery is, and how it can be used to answer questions about the world. In the [next section](A3_Data_acquisition.qmd), we'll look at the various types of satellite imagery stored in the Google Earth Engine catalog.
|
||||
@@ -1,18 +1,26 @@
|
||||
# Data Acquisition {.unnumbered}
|
||||
---
|
||||
title: Data Acquisition
|
||||
---
|
||||
|
||||
One of the main advantages of GEE is that it hosts several Petabytes of satellite imagery and other spatial data sets, [all in one place](https://developers.google.com/earth-engine/datasets). Among these are many that could prove useful to those investigating illegal mining and logging, estimating conflict-induced damage, monitoring pollution from extractive industries, conducting maritime surveillance without relying on ship transponders, verifying the locations of artillery strikes, tracking missile defense systems and many other topics.
|
||||
|
||||
|
||||
This section highlights ten categories of geospatial data available natively in the GEE catalog, ranging from optical satellite imagery, to atmospheric data, to building footprints. Each sub-section provides an overview of the given data type, suggests potential applications, and lists the corresponding datasets in the GEE catalog. The datasets listed under each heading are **not** an exhaustive list-- there are over 500 in the whole catalog, and the ones listed in this section are simply the ones with the most immediate relevance to open source investigations. If a particular geospatial dataset you want to work with isn't hosted in the GEE catalog, you can upload your own data. We'll cover that in the next section.
|
||||
|
||||
One of the main advantages of GEE is that it hosts several Petabytes of satellite imagery and other spatial data sets, [all in one place](https://developers.google.com/earth-engine/datasets). Among these are a many that could prove useful to those investigating illegal mining and logging, estimating conflict-induced damage, monitoring pollution from extractive industries, conducting maritime surveillance without relying on ship transponders, verifying the locations of artillery strikes, tracking missile defense systems, and many other topics.
|
||||
|
||||
This section highlights ten categories of geospatial data available natively in the GEE catalogue ranging from optical satellite imagery, to atmospheric data, to building footprints. Each sub-section provides an overview of the given data type, suggests potential applications, and lists the corresponding datasets in the GEE catalogue. The datasets listed under each heading are **not** an exhaustive list-- there are over 500 in the whole catalogue, and the ones listed in this section are simply the ones with the most immediate relevance to open source investigations. If a particular geospatial dataset you want to work with isn't hosted in the GEE catalog, you can upload your own data. We'll cover that in the next section.
|
||||
|
||||
|
||||
## Optical Imagery
|
||||
 tutorial.](./images/obj_det3.jpg)
|
||||
 tutorial.](../images/obj_det3.jpg)
|
||||
|
||||
|
||||
Optical satellite imagery is the bread and butter of many open source investigations. It would be tough to list off all of the possible use cases, so here's a handy flowchart:
|
||||
|
||||
Optical satellite imagery is the bread and butter of many open source investiagtions. It would be tough to list off all of the possible use cases, so here's a handy flowchart:
|
||||
|
||||
```{mermaid}
|
||||
%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#FFFFFF' ,'primaryBorderColor':'#000000' , 'lineColor':'#009933'}}}%%
|
||||
|
||||
|
||||
flowchart
|
||||
A(Does it happen outside?)
|
||||
A--> B(Yes)
|
||||
@@ -30,12 +38,16 @@ F-->G
|
||||
C-->H
|
||||
```
|
||||
|
||||
This is, of course, a bit of an exaggeration. But if you're interested in a visible phenomenon that happens outdoors and that isn't very tiny, chances are an earth-observing satellite has taken a picture of it. What that picture can tell you naturally depends on what you're interested in learning. For a deeper dive into analyzing optical satellite imagery, see the subsection on [multispectral remote sensing.](ch2.qmd#multispectral-remote-sensing-remote_sensing).
|
||||
|
||||
There are several different types of optical satellite imagery available in the GEE catalogue. The main collections are the Landsat and Sentinel series of satellites, which are operated by NASA and the European Space Agency, respectively. Landsat satellites have been in orbit since 1972, and Sentinel satellites have been in orbit since 2015. Norway's International Climate and Forest Initiative (NICFI) has also contributed to the GEE catalogue by providing a collection of optical imagery from Planet's PlanetScope satellites. These are higher resolution (4.7 meters per pixel) than Landsat (30m/px) and Sentinel-2 (10m/px), but are only available for the tropics. Even higher resolution imagery (60cm/px) is available from the GEE catalogue from the National Agriculture Imagery Program, but it is only available for the United States. For more details, see the "Datasets" section below.
|
||||
This is, of course, a bit of an exaggeration. But if you're interested in a visible phenomenon that happens outdoors and that isn't very small, chances are an earth-observing satellite has taken a picture of it. What that picture can tell you naturally depends on what you're interested in learning. For a deeper dive into analyzing optical satellite imagery, see the subsection on [multispectral remote sensing.](A2_Remote_Sensing.qmd#multispectral-remote-sensing-remote_sensing).
|
||||
|
||||
|
||||
There are several different types of optical satellite imagery available in the GEE catalog. The main collections are the Landsat and Sentinel series of satellites, which are operated by NASA and the European Space Agency, respectively. Landsat satellites have been in orbit since 1972, and Sentinel satellites have been in orbit since 2015. Norway's International Climate and Forest Initiative (NICFI) has also contributed to the GEE catalog by providing a collection of optical imagery from Planet's PlanetScope satellites. These are higher resolution (4.7 meters per pixel) than Landsat (30m/px) and Sentinel-2 (10m/px), but are only available for the tropics. Even higher resolution imagery (60cm/px) is available from the GEE catalog from the National Agriculture Imagery Program, but it is only available for the United States. For more details, see the "Datasets" section below.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Geolocating pictures
|
||||
- Some of Bellingcat's [earliest work](https://www.bellingcat.com/resources/how-tos/2014/07/09/verification-and-geolocation-tricks-and-tips-with-google-earth/) involved figuring out where a picture was taken by cross-referencing it with optical satellite imagery.
|
||||
* General surveillance
|
||||
@@ -52,8 +64,11 @@ There are several different types of optical satellite imagery available in the
|
||||
- Tracking [illegal logging](https://www.theguardian.com/environment/2016/mar/02/new-satellite-mapping-a-game-changer-against-illegal-logging) across the world.
|
||||
|
||||
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [Landsat 1-5](https://developers.google.com/earth-engine/datasets/catalog/landsat-mss) | 1972–1999 | 30m | Global |
|
||||
@@ -65,40 +80,55 @@ There are several different types of optical satellite imagery available in the
|
||||
| [NAIP](https://developers.google.com/earth-engine/datasets/catalog/USDA_NAIP_DOQQ) | 2002-2021 | 0.6m | USA |
|
||||
|
||||
|
||||
|
||||
|
||||
## Radar Imagery
|
||||
|
||||
|
||||
|
||||
|
||||
Synthetic Aperture Radar imagery (SAR) is a type of remote sensing that uses radio waves to detect objects on the ground. SAR imagery is useful for detecting objects that are small, or that are obscured by clouds or other weather phenomena. SAR imagery is also useful for detecting objects that are moving, such as ships or cars.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Change/Damage detection
|
||||
* Tracking military radar systems
|
||||
* Maritime surveillance
|
||||
* Monitoring illegal mining/logging
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [Sentinel 1](https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S1_GRD) | 2014-Present | 10m | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
## Nighttime Lights
|
||||
|
||||
|
||||
|
||||
Satellite images of the Earth at night a useful proxy for human activity. The brightness of a given area at night is a function of the number of people living there and the nature of their activities. The effects of conflict, natural disasters, and economic development can all be inferred from changes in nighttime lights.
|
||||
|
||||
The timelapse above reveals a number of interesting things: The capture of Mosul by ISIS in 2014 and the destruction of its infrastructure during the fighting (shown as the city darkening), as well as the liberation of the city by the Iraqi military in 2017 are all visible in nighttime lights. The code to create this gif, as well as a more in-depth tutorial on the uses of nighttime lights, can be found in the ["War at Night"](lights.qmd) case study.
|
||||
Satellite images of the Earth at night are a useful proxy for human activity. The brightness of a given area at night is a function of the number of people living there and the nature of their activities. The effects of conflict, natural disasters, and economic development can all be inferred from changes in nighttime lights.
|
||||
|
||||
|
||||
The timelapse above reveals a number of interesting things: The capture of Mosul by ISIS in 2014 and the destruction of its infrastructure during the fighting (shown as the city darkening), as well as the liberation of the city by the Iraqi military in 2017 are all visible in nighttime lights. The code to create this gif, as well as a more in-depth tutorial on the uses of nighttime lights, can be found in the ["War at Night"](C1_Lights.qmd) case study.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Damage detection
|
||||
* Identifying gas flaring/oil production
|
||||
* Identifying urban areas/military bases illuminated at night
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [DMSP-OLS](https://developers.google.com/earth-engine/datasets/catalog/NOAA_DMSP-OLS_NIGHTTIME_LIGHTS) | 1992-2014 | 927m | Global |
|
||||
@@ -106,84 +136,119 @@ The timelapse above reveals a number of interesting things: The capture of Mosul
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Climate and Atmospheric Data
|
||||
|
||||
|
||||
{width=100%}
|
||||
|
||||
Climate and atmospheric data can be used to track the effects of conflict on the environment. The European Space Agency's Sentinel-5p satellites measure the concentration of a number of atmospheric gases, including nitrogen dioxide, methane, and ozone. Measurements are available on a daily basis at a fairly high resolution (1km), allowing for the detection of localized sources of pollution such as oil refineries or power plants. For example, see this [Bellingcat article](https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/) in which Wim Zwijnenburg and I trace pollution to specific facilities operated by multinational oil companies in Iraq.
|
||||
|
||||
The Copernicus Atmosphere Monitoring Service (CAMS) provides similar data at a lower spatial resolution (45km), but measurements are avaialble on an hourly basis. The timelapse above utilizes CAMS data to show a sulphur dioxide plume resulting from an ISIS attack on the Al-Mishraq Sulphur Plant in Iraq. The plant was used to produce sulphuric acid, for use in fertilizers and pesticides. The attack destroyed the plant, causing a fire which burned for a month and released [21 kilotons](https://earthobservatory.nasa.gov/images/88994/sulfur-dioxide-spreads-over-iraq) of sulphur dioxide into the atmosphere per day; the largest human-made release of sulphur dioxide in history.
|
||||
{width=100%}
|
||||
|
||||
|
||||
Climate and atmospheric data can be used to track the effects of conflict on the environment. The European Space Agency's Sentinel-5p satellites measure the concentration of a number of atmospheric gasses, including nitrogen dioxide, methane and ozone. Measurements are available on a daily basis at a fairly high resolution (1km), allowing for the detection of localized sources of pollution such as oil refineries or power plants. For example, see this [Bellingcat article](https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/) in which Wim Zwijnenburg and I trace pollution to specific facilities operated by multinational oil companies in Iraq.
|
||||
|
||||
|
||||
The Copernicus Atmosphere Monitoring Service (CAMS) provides similar data at a lower spatial resolution (45km), but measurements are available on an hourly basis. The timelapse above utilizes CAMS data to show a sulfur dioxide plume resulting from an ISIS attack on the Al-Mishraq Sulphur Plant in Iraq. The plant was used to produce sulphuric acid, for use in fertilizers and pesticides. The attack destroyed the plant, causing a fire which burned for a month and released [21 kilotons](https://earthobservatory.nasa.gov/images/88994/sulfur-dioxide-spreads-over-iraq) of sulfur dioxide into the atmosphere per day; the largest human-made release of sulfur dioxide in history.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Monitoring of airborne pollution
|
||||
* Tracing pollution back to specific facilities and companies
|
||||
* Visualizing the effects of one-off environmental catastrophes
|
||||
- Nordstream 1 leak
|
||||
- ISIS setting Mishraq sulphur plant on fire
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [CAMS NRT](https://developers.google.com/earth-engine/datasets/catalog/ECMWF_CAMS_NRT) | 2016-Present | 44528m | Global |
|
||||
| [Sentinel-5p](https://developers.google.com/earth-engine/datasets/catalog/sentinel-5p) | 2018-Present | 1113m | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
## Mineral Deposits
|
||||
|
||||
|
||||
|
||||
|
||||
Mining activities often play an important role in conflict. According to an influential [study](https://www.aeaweb.org/articles?id=10.1257/aer.20150774), "the historical rise in mineral prices might explain up to one-fourth of the average level of violence across African countries" between 1997 and 2010. Data on the location of mineral deposits can be used to identify areas where mining activities are likely to be taking place, and several such datasets are available in Google Earth Engine.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Monitoring mining activity
|
||||
* Identifying areas where mining activities are likely to be taking place
|
||||
* Mapping the distribution of resources in rebel held areas in conflicts fueled by resource extraction
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [iSDA](https://developers.google.com/earth-engine/datasets/tags/isda) | 2001-2017 | 30m | Africa |
|
||||
|
||||
|
||||
|
||||
|
||||
## Fires
|
||||
|
||||
|
||||
|
||||
|
||||
Earth-observing satellites can detect "thermal anomalies" (fires) from space. NASA's Fire Information for Resource Management System (FIRMS) provides daily data on active fires in near real time, going back to the year 2000. Carlos Gonzales wrote a comprehensive [Bellingcat article](https://www.bellingcat.com/resources/2022/10/04/scorched-earth-using-nasa-fire-data-to-monitor-war-zones/) on the use of FIRMS to monitor war zones from Ukraine to Ethiopia. The map above shows that FIRMS detected fires over Eastern Ukraine trace the frontline of the war.
|
||||
|
||||
|
||||
FIRMS data are derived from the MODIS satellite, but only show the central location and intensity of a detected fire. Another MODIS product (linked in the table below) generates a monthly map of burned areas, which can be used to assess the spatial extent of fires.
|
||||
|
||||
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
* Identification of possible artillery strikes/fighting in places like Ukraine
|
||||
* Environmental warfare and "scorched earth" policies
|
||||
* Large scale arson
|
||||
- e.g. [Refugee camps burned down in Myanmar](https://citizenevidence.org/2021/02/26/using-viirs-fire-data-for-human-rights-research/)
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [FIRMS](https://developers.google.com/earth-engine/datasets/catalog/FIRMS) |2000-Present | 1000m | Global |
|
||||
| [MODIS Burned Area](https://developers.google.com/earth-engine/datasets/catalog/CIESIN_GPWv411_GPW_Population_Count) | 2000-Present | 500m | Global |
|
||||
|
||||
|
||||
## Population Density Estimates
|
||||
|
||||
|
||||
Sometimes, we may want to get an estimate the population in a specific area to ballpark how many people might be affected by a natural disaster, a counteroffensive, or a missile strike. You can't really google "what is the population in this rectangle i've drawn in Northeastern Syria?" and get a good answer. Luckily, there are several spatial population datasets hosted in GEE that let you do just that. Some, such as WorldPop, provide estimated breakdowns by age and sex as well. However, it is extremely important to bear in mind that these are **estimates**, and will **not** take into account things like conflict-induced displacement. For example, Oak Ridge National Laboratory's LandScan program has released high-resolution population data for Ukraine, but this pertains to the pre-war population distribution. The war has radically changed this distribution, so these estimates no longer reflect where people *are*. Still, this dataset could be used to roughly estimate displacement or the number of people who will need new housing.
|
||||
|
||||
## Population Density Estimates
|
||||
|
||||
|
||||
|
||||
Sometimes, we may want to get an estimate of the population in a specific area to ballpark how many people might be affected by a natural disaster, a counteroffensive or a missile strike. You can't really Google "what is the population in this rectangle I've drawn in Northeastern Syria?" and get a good answer. Luckily, there are several spatial population datasets hosted in GEE that let you do just that. Some, such as WorldPop, provide estimated breakdowns by age and sex as well. However, it is extremely important to bear in mind that these are **estimates**, and will **not** take into account things like conflict-induced displacement. For example, Oak Ridge National Laboratory's LandScan program has released high-resolution population data for Ukraine, but this pertains to the pre-war population distribution. The war has radically changed this distribution, so these estimates no longer reflect where people *are*. Still, this dataset could be used to roughly estimate displacement or the number of people who will need new housing.
|
||||
|
||||
|
||||
### Applications: {.unnumbered}
|
||||
|
||||
|
||||
* Rough estimates of civilians at risk from conflict or disaster, provided at a high spatial resolution
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [Worldpop](https://developers.google.com/earth-engine/datasets/tags/worldpop) |2000-2021 | 92m | Global |
|
||||
@@ -192,52 +257,74 @@ Sometimes, we may want to get an estimate the population in a specific area to b
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Building Footprints
|
||||
|
||||
|
||||
|
||||
|
||||
A building footprint dataset contains the two dimensional outlines of buildings in a given area. Currently, GEE hosts one building footprint dataset which covers all of Africa. In 2022, Microsoft released a free [global building footprint dataset](https://www.microsoft.com/en-us/maps/building-footprints), though to use it in Earth Engine you'll have to download it from their [GitHub page](https://github.com/Microsoft/USBuildingFootprints) and upload it manually to GEE. The same goes for OpenStreetMap (OSM), a public database of building footprints, roads, and other features that also contains useful annotations for many buildings indicating their use. [Benjamin Strick](https://www.youtube.com/watch?v=bJkV3l5Haq0) has a great youtube video on conducting investigations using OSM data.
|
||||
|
||||
|
||||
### Applications: {.unnumbered}
|
||||
|
||||
|
||||
* Joining damage estimate data with the number of buildings in an area
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Dataset | Timeframe | Coverage |
|
||||
| ----------- | ------------ | -------- |
|
||||
| [Open Buildings](https://developers.google.com/earth-engine/datasets/catalog/GOOGLE_Research_open-buildings_v2_polygons) |2022 | Africa |
|
||||
|
||||
|
||||
## Administrative Boundaries
|
||||
|
||||
|
||||
Spatial analysis often have to aggregate information over a defined area; we may want to assess the total burned area by province in Ukraine, or count the number of Saudi airstrikes by district in Yemen. For that, we need data on these administrative boundaries. GEE hosts several such datasets at the country, province, and district (or equivalent) level.
|
||||
|
||||
## Administrative Boundaries
|
||||
|
||||
|
||||
|
||||
Spatial analysis often has to aggregate information over a defined area; we may want to assess the total burned area by province in Ukraine, or count the number of Saudi airstrikes by district in Yemen. For that, we need data on these administrative boundaries. GEE hosts several such datasets at the country, province, and district (or equivalent) level.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Quick spatial calculations for different provinces/districts in a country
|
||||
- e.g. counts of conflict events by district over time
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Dataset | Timeframe | Coverage |
|
||||
| ----------- | ------------ | -------- |
|
||||
| [FAO GAUL](https://developers.google.com/earth-engine/datasets/tags/gaul) |2015 | Global |
|
||||
|
||||
|
||||
## Global Power Plant Database
|
||||
|
||||
|
||||
The Global Power Plant Database is a comprehensive, open source database of power plants around the world. It centralizes power plant data to make it easier to navigate, compare and draw insights. Each power plant is geolocated and entries contain information on plant capacity, generation, ownership, and fuel type. As of June 2018, the database includes around 28,500 power plants from 164 countries. The database is curated by the [World Resources Institude (WRI)](https://datasets.wri.org/dataset/globalpowerplantdatabase).
|
||||
|
||||
## Global Power Plant Database
|
||||
|
||||
|
||||
|
||||
The Global Power Plant Database is a comprehensive, open source database of power plants around the world. It centralizes power plant data to make it easier to navigate, compare and draw insights. Each power plant is geolocated and entries contain information on plant capacity, generation, ownership, and fuel type. As of June 2018, the database includes around 28,500 power plants from 164 countries. The database is curated by the [World Resources Institute (WRI)](https://datasets.wri.org/dataset/globalpowerplantdatabase).
|
||||
|
||||
|
||||
### Applications: {.unnumbered}
|
||||
|
||||
|
||||
* Analyzing the impact of conflict on critical infrastructure.
|
||||
- e.g. fighting in Ukraine taking place around nuclear power facilities.
|
||||
* Could be combined with the atmospheric measurements of different pollutants and the population estimates data to assess the impact of various forms of energy generation on air quality and public health.
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Dataset | Timeframe | Coverage |
|
||||
| ----------- | ------------ | -------- |
|
||||
| [GPPD](https://developers.google.com/earth-engine/datasets/catalog/WRI_GPPD_power_plants) |2018 | Global |
|
||||
| [GPPD](https://developers.google.com/earth-engine/datasets/catalog/WRI_GPPD_power_plants) |2018 | Global |
|
||||
1576
chapters/B1_Getting_Started.qmd
Normal file
1576
chapters/B1_Getting_Started.qmd
Normal file
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,6 @@
|
||||
## Image Series
|
||||
---
|
||||
title: Image Series
|
||||
---
|
||||
|
||||
One of the paradigm-changing features of Earth Engine is the ability to access decades of imagery without the previous limitation of needing to download all the data to a local disk for processing. Because remote-sensing data files can be enormous, this used to limit many projects to viewing two or three images from different periods. With Earth Engine, users can access tens or hundreds of thousands of images to understand the status of places across decades.
|
||||
|
||||
@@ -75,7 +77,7 @@ This counterexample would also have other complications—such as the need to tr
|
||||
|
||||
The first part of the filter, map, reduce paradigm is “filtering” to get a smaller ImageCollection from a larger one. As in the milk example, filters take a large set of items, limit it by some criterion, and return a smaller set for consideration. Here, filters take an ImageCollection, limit it by some criterion of date, location, or image characteristics, and return a smaller ImageCollection (Fig. F4.0.1).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As described first in Chap. F1.2, the Earth Engine API provides a set of filters for the ImageCollection type. The filters can limit an ImageCollection based on spatial, temporal, or attribute characteristics. Filters were used in Parts F1, F2, and F3 without much context or explanation, to isolate an image from an ImageCollection for inspection or manipulation. The information below should give perspective on that work while introducing some new tools for filtering image collections.
|
||||
@@ -233,9 +235,9 @@ Map.addLayer(L5EVImedian, {
|
||||
palette: ['red', 'white', 'green']
|
||||
}, 'Median EVI');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
There are many more reducers that work with an ImageCollection to produce a wide range of summary statistics. Reducers are not limited to returning only one item from the reduction. The minMax reducer, for example, returns a two-band image for each band it is given, one for the minimum and one for the maximum.
|
||||
@@ -315,7 +317,7 @@ Map.addLayer(filteredIC, {}, 'TOA image collection');
|
||||
```
|
||||
The three selected bands (which correspond to SWIR1, NIR, and Red) display a false-color image that accentuates differences between different land covers (e.g., concrete, vegetation) in Lisbon. With the Inspector tab highlighted (Fig. F4.1.1), clicking on a point will bring up the values of bands 6, 5, and 4 from each of the images. If you open the Series option, you’ll see the values through time. For the specified point and for all other points in Lisbon (since they are all enclosed in the same Landsat scene), there are 16 images gathered in 2020. By following one of the graphed lines (in blue, yellow, or red) with your finger, you should be able to count that many distinct values. Moving the mouse along the lines will show the specific values and the image dates.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
We can also show this kind of chart automatically by making use of the ui.Chart function of the Earth Engine API. The following code snippet should result in the same chart as we could observe in the Inspector tab, assuming the same pixel is clicked.
|
||||
@@ -366,7 +368,7 @@ Map.centerObject(lisbonPoint, 5);
|
||||
```
|
||||
Run the command and zoom out. If the count of images over the entire Earth is viewed, the resulting map should look like Fig. F4.1.2. The created map data may take a few minutes to fully load in.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Note the checkered pattern, somewhat reminiscent of a Mondrian painting. To understand why the image looks this way, it is useful to consider the overlapping image footprints. As Landsat passes over, each image is wide enough to produce substantial “sidelap” with the images from the adjacent paths, which are collected at different dates according to the satellite’s orbit schedule. In the north-south direction, there is also some overlap to ensure that there are no gaps in the data. Because these are served as distinct images and stored distinctly in Earth Engine, you will find that there can be two images from the same day with the same value for points in these overlap areas. Depending on the purposes of a study, you might find a way to ignore the duplicate pixel values during the analysis process.
|
||||
@@ -417,7 +419,7 @@ Map.addLayer(p30, {
|
||||
max: 0.35}, '30%');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
We can see that the resulting composite image (Fig. 4.1.3) has almost no cloudy pixels present for this area. This happens because cloudy pixels usually have higher reflectance values. At the lowest end of the values, other unwanted effects like cloud or hill shadows typically have very low reflectance values. This is why this 30th percentile composite image looks so much cleaner than the mean composite image (meanFilteredIC) calculated earlier. Note that the reducers operate per pixel: adjacent pixels are drawn from different images. This means that one pixel’s value could be taken from an image from one date, and the adjacent pixel’s value drawn from an entirely different period. Although, like the mean and median images, percentile images such as that seen in Fig. F4.1.3 never existed on a single day, composite images allow us to view Earth’s surface without the noise that can make analysis difficult.
|
||||
@@ -436,7 +438,7 @@ percentiles.map(function(p) { var image = filteredIC.reduce(ee.Reducer.percent
|
||||
```
|
||||
Note that the code adds every percentile image as a separate map layer, so you need to go to the Layers control and show/hide different layers to explore differences. Here, we can see that low-percentile composite images depict darker, low-reflectance land features, such as water and cloud or hill shadows, while higher-percentile composite images (>70% in our example) depict clouds and any other atmospheric or land effects corresponding to bright reflectance values.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Earth Engine provides a very rich API, allowing users to explore image collections to better understand the extent and variability of data in space, time, and across bands, as well as tools to analyze values stored in image collections in a frequency domain. Exploring these values in different forms should be the first step of any study before developing data analysis algorithms.
|
||||
@@ -522,7 +524,7 @@ print(yearFiltered, 'Date-filtered CHIRPS images');
|
||||
|
||||
The CHIRPS collection contains one image for every pentad. The filtered collection above is filtered to contain one year, which equates to 72 global images. If you expand the printed collection in the Console, you will be able to see the metadata for individual images; note that their date stamps indicate that they are spaced evenly every five days (Fig. F4.2.1).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Each image’s pixel values store the total precipitation during the pentad. Without aggregation to a period that matches other datasets, these layers are not very useful. For hydrological analysis, we typically need the total precipitation for each month or for a season. Let’s aggregate this collection so that we have 12 images—one image per month, with pixel values that represent the total precipitation for that month.
|
||||
@@ -606,7 +608,7 @@ print(monthlyCollection);
|
||||
```
|
||||
We have now successfully computed an aggregated collection from the source ImageCollection by filtering, mapping, and reducing, as described in Chaps. F4.0 and F4.1. Expand the printed collection in the Console and you can verify that we now have 12 images in the newly created ImageCollection (Fig. F4.2.2).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -658,7 +660,7 @@ print(chart);
|
||||
```
|
||||
The customized chart (Fig. F4.2.3) shows the typical rainfall pattern in the city of Bengaluru, India. Bengaluru has a temperate climate, with pre-monsoon rains in April and May cooling down the city and a moderate monsoon season lasting from June to September.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -793,7 +795,7 @@ var visParams = {
|
||||
Map.addLayer(composite, visParams, 'L8 Composite');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
The resulting composite (Fig. F4.3.1) has lots of clouds, especially in the western, mountainous regions of Colombia. In tropical regions, it is very challenging to generate a high-quality, cloud-free composite without first filtering images for cloud cover, even if our collection is constrained to only include images acquired during the dry season. Therefore, let’s filter our collection by the CLOUD_COVER parameter to avoid cloudy images. We will start with images that have less than 50% cloud cover.
|
||||
@@ -815,14 +817,14 @@ print('Size landsat8 collection', landsat8.size());
|
||||
print('Size landsat8FiltClouds collection', landsat8FiltClouds.size());
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
This new composite (Fig. F4.3.2) looks slightly better than the previous one, but still very cloudy. Remember to turn off the first layer or adjust the transparency to visualize only this new composite. The code prints the size of these collections, using the size function) to see how many images were left out after we applied the cloud cover threshold. (There are 1201 images in the landsat8 collection, compared to 493 in the landsat8FiltClouds collection—a lot of scenes with cloud cover greater than or equal to 50%.)
|
||||
|
||||
Try adjusting the CLOUD_COVER threshold in the landsat8FiltClouds variable to different percentages and checking the results. For example, with 20% set as the threshold (Fig. F4.3.3), you can see that many parts of the country have image gaps. (Remember to turn off the first layer or adjust its transparency; you can also set the shown parameter in the Map.addLayer function to false so the layer does not automatically load). So there is a trade-off between a stricter cloud cover threshold and data availability. Additionally, even with a cloud filter, some tiles still present a large area cover of clouds.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
This is due to persistent cloud cover in some regions of Colombia. However, a cloud mask can be applied to improve the results. The Landsat 8 Collection 2 contains a quality assessment (QA) band called QA_PIXEL that provides useful information on certain conditions within the data, and allows users to apply per-pixel filters. Each pixel in the QA band contains unsigned integers that represent bit-packed combinations of surface, atmospheric, and sensor conditions.
|
||||
@@ -846,14 +848,14 @@ var landsat8compositeMasked = landsat8FiltMasked.median().clip(country);
|
||||
Map.addLayer(landsat8compositeMasked, visParams, 'L8 composite masked');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Because we are dealing with bits, in the maskSrClouds function we utilized the bitwiseAnd and parseInt functions. These are functions that serve the purpose of unpacking the bit information. A bitwise AND is a binary operation that takes two equal-length binary representations and performs the logical AND operation on each pair of corresponding bits. Thus, if both bits in the compared positions have the value 1, the bit in the resulting binary representation is 1 (1 × 1 = 1); otherwise, the result is 0 (1 × 0 = 0 and 0 × 0 = 0). The parseInt function parses a string argument (in our case, five-character string '11111') and returns an integer of the specified numbering system, base 2.
|
||||
|
||||
The resulting composite (Fig. F4.3.4) shows masked clouds, and is more spatially exhaustive in coverage compared to previous composites (don’t forget to uncheck the previous layers). This is because, when compositing all the images into one, we are not taking cloudy pixels into account anymore; therefore, the resulting pixel is not cloud covered but an actual representation of the landscape. However, data gaps are still an issue due to cloud cover. If you do not specifically need an annual composite, a first approach is to create a two-year composite to try to mitigate the missing data issue, or to have a series of rules that allows for selecting pixels for that particular year (as in Sect. 3 below). Change the startDate variable to 2018-01-01 to include all images from 2018 and 2019 in the collection. How does the cloud-masked composite (Fig. F4.3.5) compare to the 2019 one?
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The resulting image has substantially fewer data gaps (you can zoom in to better see them). Again, if the time period is not a constraint for the creation of your composite, you can incorporate more images from a third year, and so on.
|
||||
@@ -898,14 +900,14 @@ Map.addLayer(landsat7compositeMasked,
|
||||
max: 0.2 }, 'L7 composite masked');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Note that we used bands: ['SR_B3', 'SR_B2', 'SR_B1'] to visualize the composite because Landsat 7 has different band designations. The sensors aboard each of the Landsat satellites were designed to acquire data in different ranges of frequencies along the electromagnetic spectrum. Whereas for Landsat 8, the red, green, and blue bands are B4, B3, and B2, respectively, for Landsat 7, these same bands are B3, B2, and B1, respectively.
|
||||
|
||||
You should see an image with systematic gaps like the one shown in Fig. F4.3.6 (remember to turn off the other layers, and zoom in to better see the data gaps). Landsat 7 was launched in 1999, but since 2003, the sensor has acquired and delivered data with data gaps caused by a scan line corrector (SLC) failure. Without an operating SLC, the sensor’s line of sight traces a zig-zag pattern along the satellite ground track, and, as a result, the imaged area is duplicated and some areas are missed. When the Level 1 data are processed, the duplicated areas are removed, leaving data gaps (Fig. F4.3.7). For more information about Landsat 7 and SLC error, please refer to the USGS Landsat 7 page. However, even with the SLC error, we can still use the Landsat 7 data in our composite. Now, let’s combine the Landsat 7 and 8 collections.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Since Landsat 7 and 8 have different band designations, first we create a function to rename the bands from Landsat 7 to match the names used for Landsat 8 and map that function over our Landsat 7 collection.
|
||||
@@ -924,7 +926,7 @@ var landsat7FiltMaskedRenamed = landsat7FiltMasked.map(rename);
|
||||
```
|
||||
If you print the first images of both the landsat7FiltMasked and landsat7FiltMaskedRenamed collections (Fig. F4.3.8), you will see that the bands got renamed, and not all bands got copied over (SR_ATMOS_OPACITY, SR_CLOUD_QA, SR_B6, etc.). To copy these additional bands, simply add them to the rename function. You will need to rename SR_B6 so it does not have the same name as the new band 5.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Now we merge the two collections using the merge function for ImageCollection and mapping over a function to cast the Landsat 7 input values to a 32-bit float using the toFloat function for consistency. To merge collections, the number and names of the bands must be the same in each collection. We use the select function (Chap. F1.1) to select the Landsat 8 bands to be the same as Landsat 7’s. When creating the new Landsat 7 and 8 composite, if we did not select these 6 bands, we would get an error message for trying to composite a collection that has 6 bands (Landsat 7) with a collection that has 19 bands (Landsat 8).
|
||||
@@ -949,7 +951,7 @@ Map.addLayer(landsat78composite, visParams, 'L7 and L8 composite');
|
||||
```
|
||||
Comparing the composite generated considering both Landsat 7 and 8 to the Landsat 8-only composite, it is evident that there is a reduction in the amount of data gaps in the final result (Fig. F4.3.9). The resulting Landsat 7 and 8 image composite still has data gaps due to the presence of clouds and Landsat 7’s SLC-off data. You can try setting the center of the map to the point with latitude 3.6023 and longitude −75.0741 to see the inset example of Fig. F4.3.9.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -962,7 +964,7 @@ This section presents an Earth Engine application that enables the generation of
|
||||
:::{.callout-note}
|
||||
Code Checkpoint F43c. The book’s repository contains information about accessing the GEE-BAP interface and its related functions.
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
Once you have loaded the GEE-BAP interface (Fig. F4.3.10) using the instructions in the Code Checkpoint, you will notice that it is divided into three sections: (1) Input/Output options, (2) Pixel scoring options, and (3) Advanced parameters. Users indicate the study area, the time period for generating annual BAP composites (i.e., start and end years), and the path to store the results in the Input/Output options. Users have three options to define the study area. The Draw study area option uses the Draw a shape and Draw a rectangle tools to define the area of interest. The Upload image template option utilizes an image template uploaded by the user in TIFF format. This option is well suited to generating BAP composites that match the projection, pixel size, and extent to existing raster datasets. The Work globally option generates BAP composites for the entire globe; note that when this option is selected, complete data download is not available due to the Earth’s size. With Start year and End year, users can indicate the beginning and end of the annual time series of BAP image composites to be generated. Multiple image composites are then generated—one composite for each year—resulting in a time series of annual composites. For each year, composites are uniquely generated utilizing images acquired on the days within the specified Date range. Produced BAP composites can be saved in the indicated (Path) Google Drive folder using the Tasks tab. Results are generated in a tiled, TIFF format, accompanied by a CSV file that indicates the parameters used to construct the composite.
|
||||
@@ -979,7 +981,7 @@ Finally, there is a Landsat 7 ETM+ SLC-off penalty scoring function that de-emph
|
||||
|
||||
By default, the GEE-BAP application produces image composites using all the visible bands. The Spectral index option enables the user to produce selected spectral indices from the resulting BAP image composites. Available spectral indices include: Normalized Difference Vegetation Index (NDVI, Fig. F4.3.11), Enhanced Vegetation Index (EVI), and Normalized Burn Ratio (NBR), as well as several indices derived from the Tasseled Cap transformation: Wetness (TCW), Greenness (TCG), Brightness (TCB), and Angle (TCA). Composited indices are able to be downloaded as well as viewed on the map.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
GEE-BAP functions can be accessed programmatically, including pixel scoring parameters, as well as BAP image compositing (BAP), de-spiking (despikeCollection), data-gap infilling (infill), and displaying (ShowCollection) functions. The following code sets the scoring parameter values, then generates and displays the compositing results (Fig. F4.3.12) for a BAP composite that is de-spiked, with data gaps infilled using temporal interpolation. Copy and paste the code below into a new script.
|
||||
@@ -1021,7 +1023,7 @@ library.ShowCollection(BAPCS, startYear, endYear, colombia, false, null);
|
||||
library.AddSLider(startYear, endYear);
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -1121,19 +1123,19 @@ Change detection is the process of assessing how landscape conditions are changi
|
||||
|
||||
a)
|
||||
|
||||
|
||||
|
||||
|
||||
b)
|
||||
|
||||
|
||||
|
||||
|
||||
c)
|
||||
|
||||
|
||||
|
||||
|
||||
d)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Many change detection techniques use the same basic premise: that most changes on the landscape result in spectral values that differ between pre-event and post-event images. The challenge can be to separate the real changes of interest—those due to activities on the landscape—from noise in the spectral signal, which can be caused by seasonal variation and phenology, image misregistration, clouds and shadows, radiometric inconsistencies, variability in illumination (e.g., sun angle, sensor position), and atmospheric effects.
|
||||
@@ -1145,7 +1147,7 @@ For land cover changes that occur abruptly over large areas on the landscape and
|
||||
|
||||
For the practicum, you will select pre-event and post-event image scenes and investigate the conditions in these images in a false-color composite display. Next, you will calculate the NBR index for each scene and create a difference image using the two NBR maps. Finally, you will apply a threshold to the difference image to establish categories of changed versus stable areas (Fig. F4.4.2).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Preparing Imagery
|
||||
@@ -1196,7 +1198,7 @@ Map.addLayer(preImage, visParam, 'pre');
|
||||
Map.addLayer(postImage, visParam, 'post');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Calculating NBR
|
||||
@@ -1244,7 +1246,7 @@ The color ramp has dark blues for the lowest values, greens and oranges in the m
|
||||
|
||||
a) b) c)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Classifying Change
|
||||
@@ -1277,11 +1279,11 @@ Map.addLayer(diffClassified.selfMask(),
|
||||
```
|
||||
a)
|
||||
|
||||
|
||||
|
||||
|
||||
b)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Chapters F4.5 through F4.9 present more-advanced change detection algorithms that go beyond differencing and thresholding between two images, instead allowing you to analyze changes indicated across several images as a time series.
|
||||
@@ -1379,14 +1381,14 @@ Second, can you identify fitting parameters that allow the algorithm to capture
|
||||
|
||||
The best way to begin assessing these questions is to look at the time series of individual pixels. In Earth Engine, open and run the script that generates the GUI we have developed to easily deploy the LandTrendr algorithms. Run the script, and you should see an interface that looks like the one shown in Fig. 4.5.1.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The LandTrendr GUI consists of three panels: a control panel on the left, a reporting panel on the right, and a Map panel in the center. The control panel is where all of the functionality of the interface resides. There are several modules,and each is accessed by clicking on the double arrow to the right of the title. The Map panel defaults to a location in Oregon but can be manually moved anywhere in the world. The reporting panel shows messages about how to use functions, as well as providing graphical outputs.
|
||||
|
||||
Next, expand the “Pixel Time Series Options” function. For now, simply use your mouse to click somewhere on the map. Wait a few seconds even though it looks like nothing is happening – be patient!! The GUI has sent information to Earth Engine to run the LandTrendr algorithms at the location you have clicked, and is waiting for the results. Eventually you should see a chart appear in the reporting panel on the right. Fig. 4.5.2 shows what one pixel looks like in an area where the forest burned and began regrowth. Your chart will probably look different.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The key to success with the LandTrendr algorithm is interpreting these time series. First, let’s examine the components of the chart. The x-axis shows the year of observation. With LandTrendr, only one observation per year is used to describe the history of a pixel; later, we will cover how you control that value. The y-axis shows the spectral value of the index that is chosen. In the default mode, the Normalized Burn Ratio (as described in Chap. F4.4). Note that you also have the ability to pick more indices using the checkboxes on the control panel on the left. Note that we scale floating point (decimal) indices by 1000. Thus, an NBR value of 1.0 would be displayed as 1000.
|
||||
@@ -1399,7 +1401,7 @@ Next, translate that change into the changes of interest for the change processe
|
||||
|
||||
In the case of Fig. 4.5.2, we interpret the abrupt drop as a disturbance, and the subsequent rise of the index as regrowth or recovery (though not necessarily to the same type of vegetation).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Tip: LandTrendr is able to accept any index, and advanced users are welcome to use indices of their own design. An important consideration is knowing which direction indicates “recovery” and “disturbance” for the topic you are interested in. The algorithms favor detection of disturbance and can be controlled to constrain how quickly recovery is assumed to occur (see parameters below).
|
||||
@@ -1410,7 +1412,7 @@ One option is to change the index. Any single index is simply one view of the la
|
||||
|
||||
Another option is to change the date range. LandTrendr uses one value per year, but the value that is chosen can be controlled by the user. It’s possible that the change of interest is better identified in some seasons than others. We use a medoid image compositing approach, which picks the best single observation each year from a date range of images in an ImageCollection. In the GUI, you can change the date range of imagery used for compositing in the Image Collection portion of the LandTrendr Options menu (Fig. F4.5.4).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Change the Start Date and End Date to find a time of year when the distinction between cover conditions before and during the change process of interest is greatest.
|
||||
@@ -1447,7 +1449,7 @@ There are three post-processing steps to convert a segmented trajectory to a map
|
||||
|
||||
Theory: We’ll start with a single pixel to learn how to Interpret a disturbance pixel time series in terms of the dominant disturbance segment. For the disturbance time series we have used in figures above, we can identify the key parameters of the segment associated with the disturbance. For the example above, we have extracted the actual NBR values of the fitted time series and noted them in a table (Fig. 4.5.5). This is not part of the GUI – it is simply used here to work through the concepts.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
From the table shown in Fig. 4.5.5, we can infer several key things about this pixel:
|
||||
@@ -1464,7 +1466,7 @@ If we wanted to map the magnitude of the disturbance, we would follow the same f
|
||||
|
||||
The LandTrendr GUI provides a set of tools to easily apply the same logic rules to all pixels of interest and create maps. Click on the Change Filter Options menu. The interface shown in Fig. 4.5.6 appears.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The first two sections are used to identify the segments of interest.
|
||||
@@ -1477,12 +1479,12 @@ Each filter (magnitude, duration, etc.) is used to further winnow the possible s
|
||||
|
||||
If you’re following along and making changes, or if you’re just using the default location and parameters, click the Add Filtered Disturbance Imagery to add this to the map. You should see something like Fig. 4.5.7.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
There are multiple layers of disturbance added to the map. Use the map layers checkboxes to change which is shown. Magnitude of disturbance, for example, is a map of the delta change between beginning and endpoints of the segments (Fig. 4.5.8).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Conclusion {.unnumbered}
|
||||
@@ -1565,7 +1567,7 @@ Higher-dimension band values: array images. That more complex conception of the
|
||||
|
||||
First, we will give some very basic notation (Fig. F4.6.1). A scalar pixel at time t is given by pt, and a pixel vector by pt. A variable with a “hat” represents an estimated value: in this context, p̂t is the estimated pixel value at time t. A time series is a collection of pixel values, usually sorted chronologically: {pt; t = t0...tN}, where t might be in any units, t0 is the smallest, and tN is the largest such t in the series.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Data Preparation and Preprocessing
|
||||
@@ -1646,7 +1648,7 @@ var landsat8Chart = ui.Chart.image.series(landsat8sr.select('NDVI'), roi)
|
||||
print(landsat8Chart);
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
We can add a linear trend line to our chart using the trendlines parameters in the setOptions function for image series charts. Copy and paste the code below to print the same chart but with a linear trend line plotted (Fig. F4.6.3). In the next section, you will learn how to estimate linear trends over time.
|
||||
@@ -1667,7 +1669,7 @@ var landsat8ChartTL = ui.Chart.image.series(landsat8sr.select('NDVI'), roi)
|
||||
print(landsat8ChartTL);
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Now that we have plotted and visualized the data, lots of interesting analyses can be done to the time series by harnessing Earth Engine tools for fitting curves through this data. We will see a couple of examples in the following sections.
|
||||
@@ -1714,7 +1716,7 @@ Map.addLayer(coefficients, {}, 'coefficients image');
|
||||
```
|
||||
If you click over a point using the Inspector tab, you will see the pixel values for the array image (coefficients “t” and “constant”, and residuals) and two-band image (coefficients “t” and “constant”) (Fig. F4.6.4).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Now, copy and paste the code below to use the model to detrend the original NDVI time series and plot the time series chart with the trendlines parameter (Fig. F4.6.5).
|
||||
@@ -1740,7 +1742,7 @@ var detrendedChart = ui.Chart.image.series(detrended, roi, null, 30)
|
||||
});print(detrendedChart);
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -1802,14 +1804,14 @@ print(ui.Chart.image.series(
|
||||
}));
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Returning to the mind-bending nature of curve-fitting, it is worth remembering that the harmonic waves seen in Fig. F4.6.6 are the fit of the data to a single point across the image. Next, we will map the outcomes of millions of these fits, pixel by pixel, across the entire study area.
|
||||
|
||||
We’ll compute and map the phase and amplitude of the estimated harmonic model for each pixel. Phase and amplitude (Fig. F4.6.7) can give us additional information to facilitate remote sensing applications such as agricultural mapping and land use and land cover monitoring. Agricultural crops with different phenological cycles can be distinguished with phase and amplitude information, something that perhaps would not be possible with spectral information alone.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Copy and paste the code below to compute phase and amplitude from the coefficients and add this image to the map (Fig. F4.6.8).
|
||||
@@ -1839,7 +1841,7 @@ var rgb = ee.Image.cat([
|
||||
Map.addLayer(rgb, {}, 'phase (hue), amplitude (sat), ndvi (val)');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
The code uses the HSV to RGB transformation hsvToRgb for visualization purposes (Chap. F3.1). We use this transformation to separate color components from intensity for a better visualization. Without this transformation, we would visualize a very colorful image that would not look as intuitive as the image with the transformation. With this transformation, phase, amplitude, and mean NDVI are displayed in terms of hue (color), saturation (difference from white), and value (difference from black), respectively. Therefore, darker pixels are areas with low NDVI. For example, water bodies will appear as black, since NDVI values are zero or negative. The different colors are distinct phase values, and the saturation of the color refers to the amplitude: whiter colors mean amplitude closer to zero (e.g., forested areas), and the more vivid the colors, the higher the amplitude (e.g., croplands). Note that if you use the Inspector tool to analyze the values of a pixel, you will not get values of phase, amplitude, and NDVI, but the transformed values into values of blue, green, and red colors.
|
||||
@@ -1873,9 +1875,9 @@ Map.addLayer(anotherView,
|
||||
max: 0.03 }, 'Another combination of fit characteristics');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The upper image in Fig. F4.6.9 is a closer view of Fig. F4.6.8, showing an image that transforms the sine and cosine coefficient values, and incorporates information from the mean NDVI. The lower image draws the sine and cosine in the red and blue bands, and extracts the slope of the linear trend that you calculated earlier in the chapter, placing that in the green band. The two views of the fit are similarly structured in their spatial pattern—both show fields to the west and the city to the east. But the pixel-by-pixel variability emphasizes a key point of this chapter: that a fit to the NDVI data is done independently in each pixel in the image. Using different elements of the fit, these two views, like other combinations of the data you might imagine, can reveal the rich variability of the landscape around Modesto.
|
||||
@@ -1892,11 +1894,11 @@ Code Checkpoint F46e. The book’s repository contains a script to use to begin
|
||||
:::
|
||||
Beginning with the repository script, changing the value of the harmonics variable will change the complexity of the harmonic curve fit by superimposing more or fewer harmonic waves on each other. While fitting higher-order functions improves the goodness-of-fit of the model to a given set of data, many of the coefficients may be close to zero at higher numbers or harmonic terms. Fig. F4.6.10 shows the fit through the example point using one, two, and three harmonic curves.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Conclusion {.unnumbered}
|
||||
@@ -1973,7 +1975,7 @@ Once you have loaded the CCDC interface (Fig. F4.7.1), you will be able to navig
|
||||
|
||||
Pay special attention to the characteristics of each segment. For example, look at the average surface reflectance value for each segment. The presence of a pronounced slope may be indicative of phenomena like vegetation regrowth or degradation. The number of harmonics used in each segment may represent seasonality in vegetation (either natural or due to agricultural practices) or landscape dynamics (e.g., seasonal flooding).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Question 1. While still using the SWIR1 band, click on a pixel that is forested. What do the time series and time segments look like?
|
||||
@@ -2173,14 +2175,14 @@ Map.addLayer(selectedMag, magVisParams, 'Max mag');
|
||||
Map.addLayer(selectedTbreak, timeVisParams, 'Time of max mag');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
We first take the absolute value because the magnitudes can be positive or negative, depending on the direction of the change and the band used. For example, a positive value in the SWIR1 may show a forest loss event, where surface reflectance goes from low to higher values. Brighter values in Fig. 4.7.3 represent events of that type. Conversely, a flooding event would have a negative value, due to the corresponding drop in reflectance. Once we find the maximum absolute value, we find its position on the array and then use that index to extract the original magnitude value, as well as the time when that break occurred.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -2255,7 +2257,7 @@ Map.addLayer(intercept.arrayFlatten([
|
||||
]), intVisParams, 'INTP_SWIR1');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Since we run the CCDC algorithm on Landsat surface reflectance images, intercept values should represent the average reflectance of a segment. However, if you click on the image, you will see that the values are outside of the 0–1 range. This is because the intercept is calculated by the CCDC algorithm for the origin (e.g., time 0), and not for the year we requested. In order to retrieve the adjusted intercept, as well as other coefficients, we will use a different approach.
|
||||
@@ -2351,14 +2353,14 @@ Map.addLayer(coefs, rgbVisParams, 'RGB 2005-09-25');
|
||||
```
|
||||
The slope and RMSE images are shown in Fig. 4.7.5. For the slopes, high positive values are bright, while large negative values are very dark. Most of the remaining forest is stable and has a slope close to zero, while areas that have experienced transformation and show agricultural activity tend to have positive slopes in the RED band, appearing bright in the image. Similarly, for the RMSE image, stable forests present more predictable time series of surface reflectance that are captured more faithfully by the time segments, and therefore present lower RMSE values, appearing darker in the image. Agricultural areas present noisier time series that are more challenging to model, and result in higher RMSE values, appearing brighter.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Finally, the RGB image we created is shown in Fig. 4.7.6. The intercepts are calculated for the middle point of the time segment intercepting the date we requested, representing the average reflectance for the span of the selected segment. In that sense, when shown together as an RGB image, they are similar to a composite image for the selected date, with the advantage of always being cloud-free.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -2477,7 +2479,7 @@ Code Checkpoint F48a. The book’s repository contains a script that shows what
|
||||
To see if BULC can successfully sift through these Events, we will use BULC’s GUI (Fig. F4.8.1), which makes interacting with the functionality straightforward. :::{.callout-note}
|
||||
Code Checkpoint F48b in the book’s repository contains information about accessing that interface.
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
After you have run the script, BULC’s interface requires that a few parameters be set; these are specified using the left panel. Here, we describe and populate each of the required parameters, which are shown in red. As you proceed, the default red color will change to green when a parameter receives a value.
|
||||
@@ -2498,7 +2500,7 @@ BULC makes relatively small demands on memory since its arithmetic uses only mul
|
||||
|
||||
When you have finished setting the required parameters, the interface will look like Fig. 4.8.2.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Beneath the required parameters is a set of optional parameters that affect which intermediate results are stored during a run for later inspection. We are also given a choice of returning intermediate results for closer inspection. At this stage, you can leave all optional parameters out of the BULC call by leaving them blanked or unchecked.
|
||||
@@ -2509,9 +2511,9 @@ The BULC interpretation of the landscape looks roughly like the Event inputs, bu
|
||||
|
||||
Meanwhile, below the Console, the rest of the interface changes when BULC is run. The Map panel displays BULC’s classification for the final date: that is, after considering the evidence from each of the input classifications. We can use the Satellite background to judge whether BULC is accurately capturing the state of LULC. This can be done by unselecting the drawn layers in the map layer set and selecting Satellite from the choices in the upper-right part of the Map panel. Earth Engine’s background satellite images are often updated, so you should see something like the right side of Fig. F4.8.3, though it may differ slightly.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Question 1. When comparing the BULC classification for 2021 against the current Earth Engine satellite view, what are the similarities and differences? Note that in Earth Engine, the copyrighted year numbers at the bottom of the screen may not coincide with the precise date of the image shown.
|
||||
@@ -2529,9 +2531,9 @@ Question 2. Select the BULC option, then select the Movie tool to view the resul
|
||||
|
||||
BULC results can be viewed interactively, allowing you to view more detailed estimations of the LULC around the study area. We will zoom into a specific area where change did occur after 2016. To do that, turn on the Satellite view and zoom in. Watching the scale bar in the lower right of the Map panel, continue zooming until the scale bar says 5 km. Then, enter "-60.742, -9.844" in the Earth Engine search tool, located above the code. The text will be interpreted as a longitude/latitude value and will offer you a nearby coordinate, indicated with a value for the degrees West and the degrees South. Click that entry and Earth Engine will move to that location, while keeping at the specified zoom level. Let’s compare the BULC result in this sector against the image from Earth Engine’s satellite view that is underneath it (Fig. 4.8.4).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
BULC captured the changes between 2016 and 2021 with a classification series that suggests agricultural development (Fig. 4.8.4, left). Given the appearance of BULC’s 2021 classification, it suggests that the satellite backdrop at the time of this writing (Fig. 4.8.4, right) came from an earlier time period.
|
||||
@@ -2540,7 +2542,7 @@ Now, in the Results panel, select BULC, then Movie. Set your desired frame speed
|
||||
|
||||
With this finer-scale access to the results of BULC, you can select individual pixels to inspect. Move the horizontal divider downward to expose the Inspector tab and Console tab. Use the Inspector to click on several pixels to learn their history as expressed in the inputted Events and in BULC’s interpretation of the noise and signal in the Event series. In a chosen pixel, you might see output that looks like Fig. 4.8.5. It indicates a possible conversion in the Event time series after a few classifications of the pixel as Forest. This decreases the confidence that the pixel is still Forest (Fig. 4.8.5, lower panel), but not enough for the Active Agriculture class (class 3) to become the dominant probability. After the subsequent Event labels the pixel as Forest, the confidence (lower panel) recovers slightly, but not to its former level. The next Event classifies the pixel as Active Agriculture, confidently, by interpreting that second Active Agriculture classification, in a setting where change was already somewhat suspected after the first non-Forest classification. BULC’s label (middle panel) changes to be Active Agriculture at that point in the sequence. Subsequent Event classifications as Active Agriculture creates a growing confidence that its proper label at the end of the sequence was indeed Active Agriculture.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Question 3. Run the code again with the same data, but adjust the three levelers, then view the results presented in the Map window and the Results panel. How do each of the three parameters affect the behavior of BULC in its results? Use the thumbnail to assess your subjective satisfaction with the results, and use the Inspector to view the BULC behavior in individual pixels. Can you produce an optimal outcome for this given set of input classifications?
|
||||
@@ -2571,7 +2573,7 @@ After you have run the script to initialize the interface, BULC-D’s interface
|
||||
|
||||
Run BULC-D for this area. As a reminder, you should first zoom in enough that the scale bar reads “5 km” or finer. Then, search for the location "-60.7624, -9.8542". When you run BULC-D, a result like Fig. F4.8.6 is shown for the layer of probabilities.
|
||||
|
||||
|
||||
|
||||
|
||||
Fig. 4.8.6 Result for BULC-D for the Roosevelt River area, depicting estimated probability of change and stability for 2021
|
||||
|
||||
@@ -2581,7 +2583,7 @@ Each pixel experiences its own NBR history in both the expectation period and th
|
||||
|
||||
Figure F4.8.7 shows the NBR history for the red balloon in the southern part of the study area in Fig. F4.8.4. If you click on that pixel or one like it, you can see that, whereas the values were quite stable throughout the growing season for the years used to create the pixel’s expectation, they were persistently lower in the target year. This is flagged as a likely meaningful drop in the NBR by BULC-D, for consideration by the analyst.
|
||||
|
||||
|
||||
|
||||
|
||||
Fig. 4.8.7 NBR history for a pixel with an apparent drop in NBR in the target year (below) as compared to the expectation years (above). Pixel is colored a shade of red in Fig. 4.8.6.
|
||||
|
||||
@@ -2589,12 +2591,12 @@ Figure F4.8.8 shows the NBR history for the blue balloon in the southern part of
|
||||
|
||||
Question 4. Experiment with turning off one of the satellite sensor data sources used to create the expectation collection. For example, do you get the same results if the Sentinel-2 data stream is not used, or is the outcome different. You might make screen captures of the results to compare with Fig. 4.8.4. How strongly does each satellite stream affect the outcome of the estimate? Do differences in the resulting estimate vary across the study area?
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Figure F4.8.8 also shows that, for that pixel, the fit of values for the years used to build the expectation showed a sine wave (shown in blue), but with a fit that was not very strong. When data for the target year was assembled (Fig. F4.8.8, bottom), the values were persistently above expectation throughout the growing season. Note that this pixel was identified as being different in the target year as compared to earlier years, which does not rule out the possibility that the LULC of the area was changed (for example, from Forest to Agriculture) during the years used to build the expectation collection. BULC-D is intended to be run steadily over a long period of time, with the changes marked as they occur, after which point the expectation would be recalculated.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Fig. F4.8.9 shows the NBR history for the green balloon in the southern part of the study area in Fig. F4.8.4. For that pixel, the values in the expectation collection formed a sine wave, and the values in the target collection deviated only slightly from the expectation during the target year.
|
||||
@@ -2619,12 +2621,12 @@ Next, BULC will need to know which years of Dynamic World you are interested in.
|
||||
|
||||
When you have finished, select Apply Parameters at the bottom of the input panel. When it runs, BULC subsets the Dynamic World dataset to clip out according to the dates and location, identifying images from more than 40 distinct dates. The area covers two of the tiles in which Dynamic World classifications are partitioned to be served, so BULC receives more than 90 classifications. When BULC finishes its run, the Map panel will look like Fig. F4.8.10, BULC’s estimate of the final state of the landscape at the end of the classification sequence.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Let’s explore the suite of information returned by BULC about this time period in Dynamic World. Enter “Muiraquitã” in the search bar and view the results around that area to be able to see the changing LULC classifications within farm fields. Then, begin to inspect the results by viewing a Movie of the Events, with a data frame rate of 6 frames per second. Because the study area spans multiple Dynamic World tiles, you will find that many Event frames are black, meaning that there was no data in your sector on that particular image. Because of this, and also perhaps because of the very aggressive cloud masking built into Dynamic World, viewing Events (which, as a reminder, are the individual classified images directly from Dynamic World) can be a very challenging way to look for change and stability. BULC’s goal is to sift through those classifications to produce a time series that reflects, according to its estimation, the most likely LULC value at each time step. View the Movie of the BULC results and ask yourself whether each class is equally well replicated across the set of classifications. A still from midway through the Movie sequence of the BULC results can be seen in Fig. F4.8.11.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As BULC uses the classification inputs to estimate the state of the LULC at each time step, it also tracks its confidence in those estimates. This is shown in several ways in the interface.
|
||||
@@ -2634,7 +2636,7 @@ As BULC uses the classification inputs to estimate the state of the LULC at each
|
||||
* Another way of viewing BULC’s confidence is through the Inspector tab. You can click on individual pixels to view their values in the Event time series and in the BULC time series, and see BULC’s corresponding confidence value changing through time in response to the relative stability of each pixel’s classification.
|
||||
* Another way to view BULC’s confidence estimation is as a hillshade enhancement of the final BULC classification. If you select the Probability Hillshade in the set of Map layers, it shows the final BULC classification as a textured surface, in which you can see where lower-confidence pixels are classified.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### 5.2. Using BULC To Visualize Uncertainty of Dynamic World in Simplified Categories
|
||||
@@ -2657,7 +2659,7 @@ Before continuing, think for a moment about how many classes you have now. From
|
||||
|
||||
The confidence image shown in the main Map panel is instructive (Fig. 4.8.13). Using data from 2020, 2021, and 2022, It indicates that much of the uncertainty among the original Dynamic World classifications was in distinguishing labels within agricultural fields. When that uncertainty is removed by combining classes, the BULC result indicates that a substantial part of the remaining uncertainty is at the edges of distinct covers. For example, in the south-central and southern part of the frame, much of the uncertainty among classifications in the original Dynamic World classifications was due to distinction among the highly similar, easily confused classes. Much of what remained (right) after remapping (right) formed outlines of the river and the edges between farmland and forest: a graphic depiction of the “spatial uncertainty” discussed earlier. Yet not all of the uncertainty was spatial; the thicker, darker areas of uncertainty even after remapping (right, at the extreme eastern edge for example) indicates a more fundamental disagreement in the classifications. In those pixels, even when the Agriculture-like classes were compressed, there was still considerable uncertainty (likely between Forest and Active Agriculture) in the true state of these areas. These might be of further interest: were they places newly deforested in 2020-2022? Were they abandoned fields regrowing? Were they degraded at some point? The mapping of uncertainty may hold promise for a better understanding of uncertainty as it is encountered in real classifications, thanks to Dynamic World.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Given the tools and approaches presented in this lab, you should now be able to import your own classifications for BULC (Sects. 1–3), detect changes in sets of raw imagery (Sect. 4), or use Dynamic World’s pre-created classifications (Sect. 5). The following exercises explore this potential.
|
||||
@@ -2920,7 +2922,7 @@ Map.addLayer(covariance17.arrayGet([0, 1]),
|
||||
```
|
||||
Inspect the pixel values of the resulting covariance image (Fig. F4.9.1). The covariance is positive when the greater values of one variable (at time t) mainly correspond to the greater values of the other variable (at time h), and the same holds for the lesser values, therefore, the values tend to show similar behavior. In the opposite case, when the greater values of a variable correspond to the lesser values of the other variable, the covariance is negative.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The diagonal elements of the variance-covariance array are variances. Copy and paste the code below to define and map a function to compute correlation (Fig. F4.9.2) from the variance-covariance array.
|
||||
@@ -2944,7 +2946,7 @@ Map.addLayer(correlation17,
|
||||
},
|
||||
'correlation (lag = 17 days)');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
Higher positive values indicate higher correlation between the elements of the dataset, and lower negative values indicate the opposite.
|
||||
@@ -2983,7 +2985,7 @@ Map.addLayer(corr1PrecipNDVI, {
|
||||
```
|
||||
What do you observe from this result? Looking at the cross-correlation image (Fig. F4.9.3), do you observe high values where you would expect high NDVI values (vegetated areas)? One possible drawback of this computation is that it's only based on five days of precipitation, whichever five days came right before the NDVI image.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Perhaps precipitation in the month before the observed NDVI is relevant? Copy and paste the code below to test the 30-day lag idea.
|
||||
@@ -3015,7 +3017,7 @@ Observe that the only change is to the merge method. Instead of merging the band
|
||||
|
||||
Which changes do you notice between the cross-correlation images—5 days lag vs. 30 days lag (Fig. F4.9.4)?. You can use the Inspector tool to assess if the correlation increased or not at vegetated areas.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As long as there is sufficient temporal overlap between the time series, these techniques could be extended to longer lags and longer time series.
|
||||
@@ -3093,7 +3095,7 @@ print(ui.Chart.image.series(
|
||||
}));
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
At this stage, note that the missing data has become a real problem. Any data point for which at least one of the previous points is masked or missing is also masked.
|
||||
@@ -3117,4 +3119,3 @@ Karthikeyan R, Rupner RN, Koti SR, et al (2021) Spatio-temporal and time series
|
||||
Sazib N, Bolten J, Mladenova I (2020) Exploring spatiotemporal relations between soil moisture, precipitation, and streamflow for a large set of watersheds using Google Earth Engine. Water (Switzerland) 12:1371. https://doi.org/10.3390/w12051371
|
||||
|
||||
Shumway RH, Stoffer DS (2019) Time Series: A Data Analysis Approach Using R. Chapman and Hall/CRC
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
## Vectors and Tables
|
||||
|
||||
---
|
||||
title: Vectors and Tables
|
||||
---
|
||||
|
||||
|
||||
|
||||
@@ -63,7 +64,7 @@ To demonstrate how geometry tools in Earth Engine work, let’s start by creatin
|
||||
|
||||
Click on the geometry tools in the top left of the Map pane and create a point feature. Place a new point where USF is located (see Fig. F5.0.1).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Use Google Maps to search for “Harney Science Center” or “Lo Schiavo Center for Science.” Hover your mouse over the Geometry Imports to find the +new layer menu item and add a new layer to delineate the boundary of a building on campus.
|
||||
@@ -72,7 +73,7 @@ Next, create another new layer to represent the entire campus as a polygon.
|
||||
|
||||
After you create these layers, rename the geometry imports at the top of your script. Name the layers usf_point, usf_building, and usf_campus. These names are used within the script shown in Fig. F5.0.2.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -102,7 +103,7 @@ There are many image collections loaded in Earth Engine, and they can cover a ve
|
||||
|
||||
Use your internet searching skills to locate the “Analysis Neighborhoods” dataset covering San Francisco. This data might be located in a number of places, including DataSF, the City of San Francisco’s public-facing data repository.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
After you find the Analysis Neighborhoods layer, click Export and select Shapefile (Fig. F5.0.3). Keep track of where you save the zipped file, as we will load this into Earth Engine. Shapefiles contain vector-based data—points, lines, polygons—and include a number of files, such as the location information, attribute information, and others.
|
||||
@@ -113,14 +114,14 @@ Extract the folder to your computer. When you open the folder, you will see that
|
||||
|
||||
Navigate to the Assets tab (near Scripts). Select New > Table Upload > Shape files (Fig. F5.0.4).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Select Files and Name Asset
|
||||
|
||||
Click the Select button and then use the file navigator to select the component files of the shapefile structure (i.e., .shp, .dbf, .shx, and .prj) (Fig. F5.0.5). Assign an Asset Name so you can recognize this asset.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Uploading the asset may take a few minutes. The status of the upload is presented under the Tasks tab. After your asset has been successfully loaded, click on the asset in the Assets folder and find the collection ID. Copy this text and use it to import the file into your Earth Engine analysis.
|
||||
@@ -299,7 +300,7 @@ Export.table.toDrive({
|
||||
```
|
||||
When you run this code, you will notice that you have the Tasks tab highlighted on the top right of the Earth Engine Code Editor (Fig. F5.0.6). You will be prompted to name the directory when exporting the data.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
After you run the task, the file will be saved to your Google Drive. You have now brought a feature into Earth Engine and also exported data from Earth Engine.
|
||||
@@ -456,13 +457,13 @@ var elevationDrawn = elevationVector.draw({
|
||||
Map.addLayer(elevationDrawn, {}, 'Elevation zone polygon');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You may have realized that polygons consist of complex lines, including some small polygons with just one pixel. That happens when there are no surrounding pixels of the same elevation zone. You may not need a vector map with such details—if, for instance, you want to produce a regional or global map. We can use a morphological reducer focalMode to simplify the shape by defining a neighborhood size around a pixel. In this example, we will set the kernel radius as four pixels. This operation makes the resulting polygons look much smoother, but less precise (Fig. F5.1.2).
|
||||
@@ -500,18 +501,18 @@ Map.addLayer(smoothDrawn, {}, 'Elevation zone polygon (smooth)');
|
||||
|
||||
We can see now that the polygons have more distinct shapes with many fewer small polygons in the new map (Fig. F5.1.2). It is important to note that when you use methods like focalMode (or other, similar methods such as connectedComponents and connectedPixelCount), you need to reproject according to the original image in order to display properly with zoom using the interactive Code Editor.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Raster to Points
|
||||
|
||||
Lastly, we will convert a small part of this elevation image into a point vector dataset. For this exercise, we will use the same example and build on the code from the previous subsection. This might be useful when you want to use geospatial data in a tabular format in combination with other conventional datasets such as economic indicators (Fig. F5.1.3).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The easiest way to do this is to use sample while activating the geometries parameter. This will extract the points at the centroid of the elevation pixel.
|
||||
@@ -570,7 +571,7 @@ Map.addLayer(elevationSamplesStratified, {}, 'Stratified samples');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -636,7 +637,7 @@ Map.setOptions('SATELLITE');
|
||||
```
|
||||
This will display the boundary of the La Paya protected area and deforestation in the region (Fig. F5.1.5).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
We can use Earth Engine to convert the deforestation raster to a set of polygons. The deforestation data are appropriate for this transformation as each deforestation event is labeled categorically by year, and change events are spatially contiguous. This is performed in Earth Engine using the ee.Image.reduceToVectors method, as described earlier in this section.
|
||||
@@ -667,9 +668,9 @@ Map.addLayer(deforestationVectorOutline, {
|
||||
```
|
||||
Fig. F5.1.6 shows a comparison of the raster versus vector representations of deforestation within the protected area.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Having converted from raster to vector, a new set of operations becomes available for post-processing the deforestation data. We might, for instance, be interested in the number of individual change events each year (Fig. F5.1.7):
|
||||
@@ -689,7 +690,7 @@ var chart = ui.Chart.feature
|
||||
});print(chart);
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
There might also be interest in generating point locations for individual change events (e.g., to aid a field campaign):
|
||||
@@ -910,11 +911,11 @@ Map.addLayer(distance.updateMask(protectedAreaRaster.unmask()
|
||||
max: 20000}, 'Distance outside protected area');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Sometimes it makes sense to work with objects in raster imagery. This is an unusual case of vector-like operations conducted with raster data. There is a good reason for this where the vector equivalent would be computationally burdensome.
|
||||
@@ -949,13 +950,13 @@ Map.addLayer(deforestation5km, {
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Lastly, we can estimate the deforestation area within 1 km of the protected area but only outside of the boundary.
|
||||
@@ -1253,10 +1254,10 @@ Map.addLayer(ptsTopo, {
|
||||
```
|
||||
The result is a copy of the buffered point feature collection with new properties added for the region reduction of each selected image band according to the given reducer. A part of the FeatureCollection is shown in Fig. F5.2.1. The data in that FeatureCollection corresponds to a table containing the information of Table F5.2.3. See Fig. F5.2.2 for a graphical representation of the points and the topographic data being summarized.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Table F5.2.3 Example output from zonalStats organized as a table. Rows correspond to collection features and columns are feature properties. Note that elevation and slope values in this table are rounded to the nearest tenth for brevity.
|
||||
@@ -1552,7 +1553,7 @@ Map.addLayer(pixelsFc, {
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -1644,7 +1645,7 @@ Map.addLayer(sfBlocks, {
|
||||
color: '#de2d26'}, 'Census Blocks (single color)');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
The census blocks table has a property named 'pop10' containing the population totals as of the 2010 census. We can use this to create a choropleth map showing population density. We first need to compute the population density for each feature and add it as a property. To add a new property to each feature, we can map a function over the FeatureCollection and calculate the new property called 'pop_density'. Earth Engine provides the area function, which can calculate the area of a feature in square meters. We convert it to square miles and calculate the population density per square mile.
|
||||
@@ -1684,7 +1685,7 @@ var visParams = {
|
||||
Map.addLayer(sfBlocksPaint.clip(geometry), visParams, 'Population Density');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Creating a Categorical Map
|
||||
@@ -1712,9 +1713,9 @@ var sfRoadsDraw = sfRoads.draw({
|
||||
Map.addLayer(sfRoadsDraw, {}, 'Roads (Draw)');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The road layer has a column called “MTFCC” (standing for the MAF/TIGER Feature Class Code). This contains the road priority codes, representing the various types of roads, such as primary and secondary. We can use this information to render each road segment according to its priority. The draw function doesn’t allow us to specify different styles for each feature. Instead, we need to make use of the style function.
|
||||
@@ -1747,7 +1748,7 @@ var sfRoadsStyle = sfRoads.style({
|
||||
Map.addLayer(sfRoadsStyle.clip(geometry), {}, 'Roads (Style)');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -1806,7 +1807,7 @@ var interstateRoadsDrawn = interstateRoads.draw({
|
||||
Map.addLayer(interstateRoadsDrawn, {}, 'Interstate Roads');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Let’s define a join that will select all the features from the sfBlocks layer that are within 1 km of any feature from the interstateRoads layer. We start by defining a filter using the ee.Filter.withinDistance filter. We want to compare the geometries of features in both layers, so we use a special property called '.geo' to compare the collections. By default, the filter will work with exact distances between the geometries. If your analysis does not require a very precise tolerance of spatial uncertainty, specifying a small non-zero maxError distance value will help speed up the spatial operations. A larger tolerance also helps when testing or debugging code so you can get the result quickly instead of waiting longer for a more precise output.
|
||||
@@ -1840,7 +1841,7 @@ var closeBlocksDrawn = closeBlocks.draw({
|
||||
Map.addLayer(closeBlocksDrawn, {}, 'Blocks within 1km');
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Spatial Joins
|
||||
@@ -1873,7 +1874,7 @@ var sfTreesStyled = sfTrees.style({
|
||||
Map.addLayer(sfTreesStyled, {}, 'SF Trees');
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
To find the tree points in each neighborhood polygon, we will use an ee.Filter.intersects filter.
|
||||
@@ -1903,7 +1904,7 @@ print(joined.first());
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
You will see that each feature of the sfNeighborhoods collection now has an additional property called trees. This contains all the features from the sfTrees collection that were matched using the intersectFilter. We can now map a function over the results and post-process the collection. As our analysis requires the computation of the total number of trees in each neighborhood, we extract the matching features and use the size function to get the count (Fig. F5.3.9).
|
||||
@@ -1916,7 +1917,7 @@ var sfNeighborhoods = joined.map(function(f) { var treesWithin = ee.List(f.get
|
||||
print(sfNeighborhoods.first());
|
||||
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
The results now have a property called total_trees containing the count of intersecting trees in each neighborhood polygon.
|
||||
@@ -1937,7 +1938,7 @@ Export.table.toDrive({
|
||||
```
|
||||
The final result is a CSV file with the neighborhood names and total numbers of trees counted using the join (Fig. F5.3.10).
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::{.callout-note}
|
||||
@@ -1,19 +1,29 @@
|
||||
# War at Night {.unnumbered}
|
||||
---
|
||||
title: War at Night
|
||||
format:
|
||||
html:
|
||||
number-sections: true
|
||||
number-offset: [1,5]
|
||||
---
|
||||
|
||||
## Data
|
||||
|
||||
Satellite images of Syria taken at night capture a subtle trace left by human civilization: lights. Apartment buildings, street lights, highways, powerplants-- all are illuminated at night and can be seen from space. Researchers often use these nighttime lights signatures to track development; as cities grow, villages recieve power, and infrastructure is built, areas emit more light. But this works both ways. As cities are demolished, villages burned, and highways cutoff, they stop emitting lights.
|
||||
Satellite images of Syria taken at night capture a subtle trace left by human civilization: lights. Apartment buildings, street lights, highways, power plants -- all are illuminated at night and can be seen from space. Researchers often use these nighttime lights signatures to track development; as cities grow, villages receive power and infrastructure is built, areas emit more light. But this works both ways. As cities are demolished, villages burned and highways cutoff, they stop emitting lights.
|
||||
|
||||
|
||||
In this tutorial, we'll use satellite images of Iraq taken at night to track the destruction caused by the fight against the Islamic State. We'll use the VIIRS nighttime lights dataset, which is a collection of satellite images taken by the Visible Infrared Imaging Radiometer Suite (VIIRS) on the Suomi NPP satellite. VIIRS is a sensor that can detect light in the visible and infrared spectrum, and is capable of taking images at night. A link to the GEE code for this section can be found [here](https://code.earthengine.google.com/2cf77d8cb9afd76b73100637fbffdf5d).
|
||||
|
||||
### Pre-Processing
|
||||
|
||||
First, let's start by importing a few useful packages written by [Gennadii Donchyts](https://twitter.com/gena_d). We'll use `utils` and `text` to annotate the date of each image on the timelapse. We'll also define an Area of Interest (AOI), which is just a rectangle. You can do this manually by clicking the drawing tools in the top left. I've drawn an AOI over the area covering Mosul, Irbil, and Kirkuk in Northern Iraq.
|
||||
## Pre-Processing
|
||||
|
||||
|
||||
First, let's start by importing a few useful packages written by [Gennadii Donchyts](https://twitter.com/gena_d). We'll use `utils` and `text` to annotate the date of each image on the timelapse. We'll also define an Area of Interest (AOI), which is just a rectangle. You can do this manually by clicking the drawing tools in the top left. I've drawn an AOI over the area covering Mosul, Irbil and Kirkuk in Northern Iraq.
|
||||
|
||||
|
||||
``` js
|
||||
var utils = require("users/gena/packages:utils");
|
||||
var text = require("users/gena/packages:text");
|
||||
|
||||
|
||||
// define the Area of Interest (AOI)
|
||||
var AOI = ee.Geometry.Polygon(
|
||||
[[[42.555362833405326, 36.62010778397765],
|
||||
@@ -21,10 +31,12 @@ var AOI = ee.Geometry.Polygon(
|
||||
[44.681217325592826, 35.18296243288332],
|
||||
[44.681217325592826, 36.62010778397765]]])
|
||||
|
||||
|
||||
// start and end dates for our gif
|
||||
var startDate = '2013-01-01';
|
||||
var endDate = '2018-01-01';
|
||||
|
||||
|
||||
// a filename for when we export the gif
|
||||
var export_name='qayyarah_viirs'
|
||||
|
||||
@@ -39,12 +51,15 @@ var viirs_palette = [
|
||||
"#fcffa4",
|
||||
];
|
||||
|
||||
|
||||
// Visualisation parameters for the VIIRS imagery, defining a minimum and maximum value, and referencing the palette we just created
|
||||
var VIIRSvis = { min: -0.1, max: 1.6, palette: viirs_palette };
|
||||
```
|
||||
|
||||
|
||||
Next, we'll load the VIIRS nighttime lights imagery. We want to select the `avg_rad` band of the image collection, and filter blank images. Sometimes, we get blank images over an area in VIIRS if our AOI is on the edge of the satellite's imaging swath. We can filter these images, similarly to how we filter for cloudy images in Sentinel-2:
|
||||
|
||||
|
||||
``` js
|
||||
var VIIRS= ee.ImageCollection("NOAA/VIIRS/DNB/MONTHLY_V1/VCMCFG")
|
||||
.select('avg_rad')
|
||||
@@ -68,28 +83,36 @@ var VIIRS= ee.ImageCollection("NOAA/VIIRS/DNB/MONTHLY_V1/VCMCFG")
|
||||
```
|
||||
Let's have a look at the first image in the collection to make sure everything's looking right. We'll set the basemap to satellite and center our AOI:
|
||||
|
||||
|
||||
``` js
|
||||
Map.setOptions('HYBRID')
|
||||
Map.centerObject(AOI)
|
||||
Map.addLayer(VIIRS.first(),VIIRSvis,'Nighttime Lights')
|
||||
```
|
||||
|
||||
|
||||
|
||||
If we decrease the opacity of the VIIRS layer, we can see the cities of Mosul, Erbil, and Kirkuk shining brightly at night. We can also see a string of bright lights between Kirkuk and Erbil-- these are methane flares from oil wells.
|
||||
|
||||
### Analysis
|
||||
If we decrease the opacity of the VIIRS layer, we can see the cities of Mosul, Erbil and Kirkuk shining brightly at night. We can also see a string of bright lights between Kirkuk and Erbil -- these are methane flares from oil wells.
|
||||
|
||||
|
||||
## Analysis
|
||||
|
||||
|
||||
Having pre-processed the VIIRS imagery, we can now define a function `gif` that will take:
|
||||
|
||||
|
||||
1. An image collection (`col`, in this case the nighttime lights imagery `VIIRS`)
|
||||
2. Visualization parameters (`col_vis`, in this case `VIIRSvis`)
|
||||
3. An Area of Interest `AOI`
|
||||
|
||||
|
||||
The function will then return a timelapse.
|
||||
|
||||
|
||||
``` js
|
||||
var gif = function (col, col_vis, AOI) {
|
||||
|
||||
|
||||
// Define the date annotations to be printed in the top left of the gif in white
|
||||
var annotations = [
|
||||
{
|
||||
@@ -103,6 +126,7 @@ var gif = function (col, col_vis, AOI) {
|
||||
},
|
||||
];
|
||||
|
||||
|
||||
// Next, we want to map over the image collection,
|
||||
var rgbVis = col.map(function (image) {
|
||||
// Get the date of the image and format it
|
||||
@@ -113,11 +137,13 @@ var gif = function (col, col_vis, AOI) {
|
||||
return image.visualize(col_vis).set({ label: label });
|
||||
});
|
||||
|
||||
// Now we use the label proprty and the annotateImage function from @gena_d to annotate each image with the date.
|
||||
|
||||
// Now we use the label property and the annotateImage function from @gena_d to annotate each image with the date.
|
||||
rgbVis = rgbVis.map(function (image) {
|
||||
return text.annotateImage(image, {}, AOI, annotations);
|
||||
});
|
||||
|
||||
|
||||
// Define GIF visualization parameters.
|
||||
var gifParams = {
|
||||
maxPixels: 27017280,
|
||||
@@ -127,6 +153,7 @@ var gif = function (col, col_vis, AOI) {
|
||||
framesPerSecond: 5,
|
||||
};
|
||||
|
||||
|
||||
// Export the gif to Google Drive
|
||||
Export.video.toDrive({
|
||||
collection: rgbVis, // the image collection
|
||||
@@ -138,28 +165,36 @@ var gif = function (col, col_vis, AOI) {
|
||||
// Print the GIF URL to the console.
|
||||
print(rgbVis.getVideoThumbURL(gifParams));
|
||||
|
||||
|
||||
// Render the GIF animation in the console.
|
||||
print(ui.Thumbnail(rgbVis, gifParams));
|
||||
};
|
||||
```
|
||||
Ok that was a pretty big chunk of code. But the good news is that we basically never have to touch it again, since we can just feed it different inputs. For example, if I want to generate a gif of nighttime lights over a different area, it's as simple as dragging the AOI. If I want to look at a different time period, I can just edit the `startDate` and `endDate` variables. And if I want to visualize an entirely different type of satellite imagery-- Sentinel-1, Sentinel-2, or anything else, all I have to do is change the image collection (`col`) and visualization parameters (`col_vis`) variables. Now, let's look at some timelapses.
|
||||
Ok that was a pretty big chunk of code. But the good news is that we basically never have to touch it again, since we can just feed it different inputs. For example, if I want to generate a gif of night time lights over a different area, it's as simple as dragging the AOI. If I want to look at a different time period, I can just edit the `startDate` and `endDate` variables. And if I want to visualize an entirely different type of satellite imagery -- Sentinel-1, Sentinel-2, or anything else, all I have to do is change the image collection (`col`) and visualization parameters (`col_vis`) variables. Now, let's look at some timelapses.
|
||||
|
||||
|
||||
### The Fall of Mosul
|
||||
|
||||
#### The Fall of Mosul
|
||||
|
||||
The function returns a timelapse of nighttime lights over Northern Iraq:
|
||||
``` js
|
||||
gif(VIIRS, VIIRSvis, AOI);
|
||||
```
|
||||
 and [ezgif](https://ezgif.com/) for the finishing touches. ](./images/Figure_1.gif)
|
||||
 and [ezgif](https://ezgif.com/) for the finishing touches. ](../images/Figure_1.gif)
|
||||
|
||||
|
||||
This timelapse gives a play-by-play of one of the most important campaigns in the war against the Islamic State. In the first few frames, Mosul is under the control of the Kurdistan Regional Government (KRG). In the summer of 2014, ISIS captures the city, and power is cut off. Mosul and many villages along the Tigris river are plunged into darkness. In 2015, the front line in the campaign to retake the city emerges around Mosul, advancing in 2016 and 2017. Mosul is eventually retaken by the KRG in 2017, after which it brightens once again as electricity is restored.
|
||||
|
||||
#### The Qayyarah Fires
|
||||
|
||||
### The Qayyarah Fires
|
||||
|
||||
|
||||
Farther south, there is an interesting detail. Above the "h" in "Qayyarah", a bright set of lights emerges just before Mosul is recaptured, around December 2016. Fleeing Islamic State fighters [set fire to the Qayyarah oilfields](https://time.com/iraq-fires/), which burned for months.
|
||||
|
||||
|
||||
Using the VIIRS data we've already loaded, we can further analyze the effect of the conflict using a chart. First, let's define two rectangles (again, you can draw these) over Mosul and Qayyarah:
|
||||
|
||||
|
||||
``` js
|
||||
var mosul = ee.Feature(
|
||||
ee.Geometry.Polygon(
|
||||
@@ -172,6 +207,7 @@ var mosul = ee.Feature(
|
||||
"system:index": "0"
|
||||
}),
|
||||
|
||||
|
||||
qayyarah = ee.Feature(
|
||||
ee.Geometry.Polygon(
|
||||
[[[43.08240275545117, 35.8925587996721],
|
||||
@@ -183,11 +219,13 @@ var mosul = ee.Feature(
|
||||
"system:index": "0"
|
||||
})
|
||||
|
||||
|
||||
// Let's put these together in a list
|
||||
var regions=[qayyarah, mosul]
|
||||
```
|
||||
Once we've got the rectangles, we can make a chart that will take the mean value of the VIIRS images in each rectangle over time:
|
||||
|
||||
|
||||
``` js
|
||||
var chart =
|
||||
ui.Chart.image
|
||||
@@ -202,40 +240,57 @@ var chart =
|
||||
|
||||
print(chart)
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
We can clearly see Mosul (the red line) darkening in 2014 as the city is taken by ISIS. During this period the Qayyarah oil fields are, as we might expect, quite dark. All of a sudden in 2016 Qayyarah becomes brighter at night than the city of Mosul ever was, as the oilfields are set on fire. Then, almost exactly when the blaze in Qayyarah is extinguished and the area darkens (i.e. when the blue line falls back to near zero), Mosul brightens once again (i.e. the red line rises) as the city is liberated.
|
||||
|
||||
We can clearly see Mosul (the red line) darkening in 2014 as the city is taken by ISIS. During this period the Qayyarah oilfileds are, as we might expect, quite dark. All of a sudden in 2016 Qayyarah becomes brighter at night than the city of Mosul ever was, as the oilfields are set on fire. Then, almost exactly when the blaze in Qayyarah is extinguished and the area darkens (i.e. when the blue line falls back to near zero), Mosul brightens once again (i.e. the red line rises) as the city is liberated.
|
||||
|
||||
<!--
|
||||
### The Battle for Aleppo
|
||||
|
||||
The images below were taken between 2012 and 2014. Vast swaths of the city darken as neighbourhoods are razed by fighting.
|
||||
|
||||
The images below were taken between 2012 and 2014. Vast swaths of the city darken as neighborhoods are razed by fighting.
|
||||
|
||||
|
||||
<timelapse>
|
||||
|
||||
Though this is a trend that can be observed across the country, nowhere is the decline in nightlights more visible than in Aleppo. Below is a comparison of longitudinal trends in nighlights signatures between several cities:
|
||||
|
||||
Though this is a trend that can be observed across the country, nowhere is the decline in nightlights more visible than in Aleppo. Below is a comparison of longitudinal trends in nightlights signatures between several cities:
|
||||
|
||||
|
||||
<graph>
|
||||
|
||||
The most salient trend is Aleppo plummeting over the course of 2012, and becoming steadily darker over the course of the next four years. Raqqa drops in 2012 as well, but remains in flux until 2017, when the battle to reclaim the city pluges it into near total darkness. Damascus also experiences a dip in 2012, but stabilizes relatively quickly. The Turkish city of Gaziantep-- less than 100km from Aleppo and roughly 1/5th the size-- stands in stark contrast to the Syrian cities, becoming progressively brighter over the entire period.
|
||||
|
||||
The most salient trend is Aleppo plummeting over the course of 2012, and becoming steadily darker over the course of the next four years. Raqqa drops in 2012 as well, but remains in flux until 2017, when the battle to reclaim the city plunges it into near total darkness. Damascus also experiences a dip in 2012, but stabilizes relatively quickly. The Turkish city of Gaziantep -- less than 100km from Aleppo and roughly 1/5th the size -- stands in stark contrast to the Syrian cities, becoming progressively brighter over the entire period.
|
||||
|
||||
|
||||
Another interesting pattern here is the difference in seasonal trends in nightlights. Under normal circumstances in this part of the world, cities become brighter at night during the summer months. Restaurants, bars, and markets stay open later and conduct business outdoors. Gaziantep, which still attracts scores of tourists every year, displays pronounced seasonality. Damascus, the most stable of the three Syrian cities, also maintains a seasonal trend throughout the war. In contrast, both Raqqa and Aleppo maintain extremely low and roughly constant levels of nightlights year-round during the periods following intense fighting.
|
||||
|
||||
|
||||
Reliable economic data for Syria haven't been available for nearly a decade, and assessing the country's recovery is consequently difficult. But subtle indications of economic growth are visible above: all three Syrian cities have been on a steady upward trend since 2017, and beginning to display seasonal variation once again. -->
|
||||
|
||||
|
||||
<!-- ### Fighting for Oil
|
||||
|
||||
|
||||
Throughout the war, sudden massive spikes in nightlights signatures can be observed throughout the country. In the center of the map just west of Palmyra, some particularly large spikes occur in 2017:
|
||||
|
||||
These flashes of light show gas wells being set on fire, a common form of sabotage carried out by retreating Islamic State fighters. Modified Sentinel-2 imagery of the Hayyan gas field (indicated by the green box above) shows this in greater detail. Substituing the Red band in an RGB image with Near Infrared (NIR) highlights thermal signatures, showing fires burning brightly even during the day.
|
||||
|
||||
The large complex on the right is the Hayyan Gas Plant, which produced nearly 1/3 of Syria's electricity. The plant and its associated wells changed hands several times throughout the war, but were under Islamic State control until February 2017. In the video below, Islamic State fighters can be seen rigging the plant with explosives and destroying it on January 8th:
|
||||
These flashes of light show gas wells being set on fire, a common form of sabotage carried out by retreating Islamic State fighters. Modified Sentinel-2 imagery of the Hayyan gas field (indicated by the green box above) shows this in greater detail. Substituting the Red band in an RGB image with Near Infrared (NIR) highlights thermal signatures, showing fires burning brightly even during the day.
|
||||
|
||||
|
||||
The large complex on the right is the Hayyan Gas Plant, which produced nearly one third of Syria's electricity. The plant and its associated wells changed hands several times throughout the war, but were under Islamic State control until February 2017. In the video below, Islamic State fighters can be seen rigging the plant with explosives and destroying it on January 8th:
|
||||
|
||||
|
||||
In February, three Russian oil and gas companies (Zarubij Naft, Lukoil and Gazprom Neft) were given restoration, exploration and production rights to the hydrocarbon deposits West of Palmyra. On January 12th, 2017, the Syrian Army's 5th Legion and Russian special forces launched a counterattack known as the "Palmyra offensive", with the aim of retaking several important hydrocarbon deposits including Hayyan.
|
||||
|
||||
In February, three Russian oil and gas companies (Zarubij Naft, Lukoil and Gazprom Neft) were given restoration, exploration, and production rights to the hydrocarbon deposits West of Palmyra. On January 12th, 2017, the Syrian Army's 5th Legion and Russian special forces launched a counterattack known as the "Palmyra offensive", with the aim of retaking several important hydrocarbon deposits including Hayyan.
|
||||
|
||||
The timing of well fires aligns closely with a detailed timeline of the campaign.The Near Infrared Sentinel-2 image below shows the layout of the Hayyan Gas Plant and the wells in the Hayyan gas field:
|
||||
|
||||
|
||||
The Syrian Army took the Hayyan gas field on [February 4th](https://www.almasdarnews.com/article/syrian-army-liberates-hayyan-gas-fields-west-palmyra/), and retreating ISIS fighters set fire to wells 1, and 3. However, ISIS managed to briefly retake the Hayyan field on [February 7th](https://www.almasdarnews.com/article/isis-retakes-hayyan-gas-fields-new-bid-expand-west-palmyra/), setting fire to wells 2 and 4. These moments in the Palmyra Offensive are captured in NIR signatures
|
||||
|
||||
|
||||
Interestingly, despite the massive explosion caused by the bombing of the Hayyan Gas Plant, no prolonged thermal anomalies were detected over the area of the plant itself. The well fires, on the other hand, lasted for months. Below is an image of well fire at the Hayyan field taken from this [video](https://www.youtube.com/watch?v=WFe9abYyqK0); based on the nearby infrastructure and date (04/02/2017) of posting, it is likely Well-3.
|
||||
-->
|
||||
-->
|
||||
@@ -1,20 +1,28 @@
|
||||
# Refinery Identification {.unnumbered}
|
||||
---
|
||||
title: Refinery Identification
|
||||
---
|
||||
|
||||
*Topics: multispectral satellite imagery, machine learning, informal economies, war.*
|
||||
|
||||
In Syria, over a decade of war has ravaged one of its most important industries. Oil is a basic necessity for local residents who need to heat their homes and keep the lights on. It's also an important source of income for armed groups who control production; by some estimates the Islamic State was making over [$40 million](https://www.rand.org/blog/2017/10/oil-extortion-still-paying-off-for-isis.html) per month in oil revenues, and the Syrian Democratic Forces were making [$120 million](https://www.al-monitor.com/originals/2021/08/syrian-government-kurds-discuss-plans-oil-trade) per year selling oil to their adversaries, the Syrian Government. This, in turn, has made oil production facilies a frequent target of [airstrikes](https://www.gov.uk/government/publications/british-forces-air-strikes-in-iraq-monthly-list/january-2015), leading to catastrophic environmental consequences.
|
||||
|
||||
The destruction of Syria's oil infrastructure and its importance as a source of revenue for armed groups has led to a massive rise in makeshift oil extraction and refining. These makeshift refineries are often constructed by digging a large pit, lining it with a tarp, and filling it with polluted water. A furnace heats crude oil, which is run through a pipe cooled by the basin and collected in drums:
|
||||
In Syria, over a decade of war has ravaged one of its most important industries. Oil is a basic necessity for local residents who need to heat their homes and keep the lights on. It's also an important source of income for armed groups who control production; by some estimates the Islamic State was making over [$40 million](https://www.rand.org/blog/2017/10/oil-extortion-still-paying-off-for-isis.html) per month in oil revenues, and the Syrian Democratic Forces were making [$120 million](https://www.al-monitor.com/originals/2021/08/syrian-government-kurds-discuss-plans-oil-trade) per year selling oil to their adversaries, the Syrian Government. This, in turn, has made oil production facilities a frequent target of [airstrikes](https://www.gov.uk/government/publications/british-forces-air-strikes-in-iraq-monthly-list/january-2015), leading to catastrophic environmental consequences.
|
||||
|
||||
|
||||
The destruction of Syria's oil infrastructure and its importance as a source of revenue for armed groups has led to a massive rise in makeshift oil extraction and refining. These makeshift refineries are often constructed by digging a large pit, lining it with a tarp and filling it with polluted water. A furnace heats crude oil, which is run through a pipe cooled by the basin and collected in drums:
|
||||
|
||||
|
||||
|
||||
Wim Zwijnenburg wrote an excellent [Bellingcat article](https://www.bellingcat.com/news/2020/04/24/dying-to-keep-warm-oil-trade-and-makeshift-refining-in-north-west-syria/) on the subject, which you should read before going any further in this tutorial. In the article, Wim notes that these facilities "can be spotted by the ditch and the black spot at the end with oil waste residues, which blacken the soil around the furnace." These refineries also frequently leak, blackening large swaths of land around them.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Looking around Northwestern Syria, we can see agricultural fields pockmarked by these makeshift refineries (you can pan around and zoom in):
|
||||
```{python}
|
||||
#| echo: false
|
||||
|
||||
|
||||
from ipyleaflet import Map, Marker, basemaps, basemap_to_tiles
|
||||
m = Map(
|
||||
basemap=basemap_to_tiles(
|
||||
@@ -26,37 +34,51 @@ m = Map(
|
||||
m
|
||||
```
|
||||
|
||||
Previous efforts to quantify informal oil production have involved manually sifting through satellite imagery and counting them. This is a painstaking process that leaves a number of important questions unanswered. Even if we were to count all of the individual refineries, could we get an estimate of polluted area? What if we wanted to count the refineries in a new part of Syria? Or get annual or even monthly estimates of new refineries?
|
||||
|
||||
Below is an Earth Engine application that automates the detection of makeshift refineries in Northeastern Syria, using mutlispectral satellite imagery and machine learning. Blue dots represent the locations of predicted makeshift oil refineries and general oil pollution, while red areas indicate areas predicted to be contaminated by oil.
|
||||
Previous efforts to quantify informal oil production have involved manually sifting through satellite imagery and counting them. This is a painstaking process that leaves a number of important questions unanswered. Even if we were to count all of the individual refineries, could we get an estimate of the polluted area? What if we wanted to count the refineries in a new part of Syria? Or get annual or even monthly estimates of new refineries?
|
||||
|
||||
|
||||
Below is an Earth Engine application that automates the detection of makeshift refineries in Northeastern Syria, using multispectral satellite imagery and machine learning. Blue dots represent the locations of predicted makeshift oil refineries and general oil pollution, while red areas indicate areas predicted to be contaminated by oil.
|
||||
|
||||
|
||||
:::{.column-page}
|
||||
|
||||
|
||||
<iframe src='https://ollielballinger.users.earthengine.app/view/rojavaoil' width='100%' height='700px'></iframe>
|
||||
|
||||
|
||||
:::
|
||||
|
||||
You can draw an Area of Interest (AOI) and get the total number of contaminated points as well as the total number of contaminated square meters within the AOI. drawing multiple AOIs will show a running total of these statistics. It's not perfect-- it misses some refineries and falsely identifies some others-- but it is generally quite accurate; you can visually verify the results of the prediction by zooming in using the "+" button. You can toggle different layers using the "layers" tab as well. This tool could be used to get an estimate of oil production in a user-defined area, and eventually to direct cleanup efforts. The fullscreen version of the application can be found [here](https://ollielballinger.users.earthengine.app/view/rojavaoil), and the source code [here](https://code.earthengine.google.com/7a80f10412e1eb2a4d2c5d95989e70bd). This tutorial will first cover the basics of multispectral remote sensing, before moving into a step-by-step guide in the construction of this model.
|
||||
|
||||
You can draw an Area of Interest (AOI) and get the total number of contaminated points as well as the total number of contaminated square meters within the AOI. Drawing multiple AOIs will show a running total of these statistics. It's not perfect -- it misses some refineries and falsely identifies some others -- but it is generally quite accurate; you can visually verify the results of the prediction by zooming in using the "+" button. You can toggle different layers using the "layers" tab as well. This tool could be used to get an estimate of oil production in a user-defined area, and eventually to direct cleanup efforts. The fullscreen version of the application can be found [here](https://ollielballinger.users.earthengine.app/view/rojavaoil), and the source code [here](https://code.earthengine.google.com/7a80f10412e1eb2a4d2c5d95989e70bd). This tutorial will first cover the basics of multispectral remote sensing, before moving into a step-by-step guide in the construction of this model.
|
||||
|
||||
|
||||
|
||||
|
||||
# Machine Learning Workflow
|
||||
|
||||
|
||||
## Pre-Processing
|
||||
|
||||
|
||||
As always, the first step in our project will be to load and pre-process satellite imagery. For this project, we'll be using Sentinel-2 imagery. Let's load imagery from 2020-2021, filter out cloudy images, and define visualization parameters:
|
||||
|
||||
|
||||
```js
|
||||
var start='2020-04-01'
|
||||
var end='2021-07-01'
|
||||
|
||||
|
||||
var bands = ['B2', 'B3', 'B4','B5','B6','B7','B8', 'B8A','B11','B12']
|
||||
|
||||
|
||||
var sentinel = ee.ImageCollection('COPERNICUS/S2_SR')
|
||||
.filter(ee.Filter.date(start, end))
|
||||
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 10))
|
||||
.mean()
|
||||
.select(bands)
|
||||
|
||||
|
||||
var s_rgb = {
|
||||
min: 0.0,
|
||||
max: 3000,
|
||||
@@ -65,24 +87,32 @@ var s_rgb = {
|
||||
};
|
||||
```
|
||||
|
||||
When loading the Sentinel-2 imagery, I've also onlyh selected the bands that we will ultimately use in our analysis. There are a number of other bands included in the data that we don't need. I've omitted a few bands (B1, B9, B10) because they're collected at a much lower spatial resolution (60 meters) compared to the other bands.
|
||||
|
||||
When loading the Sentinel-2 imagery, I've also only selected the bands that we will ultimately use in our analysis. There are a number of other bands included in the data that we don't need. I've omitted a few bands (B1, B9, B10) because they're collected at a much lower spatial resolution (60 meters) compared to the other bands.
|
||||
|
||||
|
||||
A couple types of landcover are so readily identifiable that we can remove them with thresholds. Water and vegetation both have spectral indices; we looked at NDVI above, but there's a similar one for water called NDWI. These can be calculated from Sentinel-2 imagery as follows:
|
||||
|
||||
|
||||
```js
|
||||
var ndvi=sentinel.normalizedDifference(['B8','B4'])
|
||||
.select(['nd'],['ndvi'])
|
||||
|
||||
|
||||
var ndwi=sentinel.normalizedDifference(['B3','B8'])
|
||||
.select(['nd'],['ndwi'])
|
||||
```
|
||||
|
||||
|
||||
We use the `normalizedDifference` function and specify which bands we want to use for each index. NDVI uses the red and near infrared bands (B4 and B8), while NDWI uses bands 3 and 8. Finally, we want to rename the resulting band from 'nd' to the name of the spectral index.
|
||||
|
||||
Now we can use these indices to filter out water and vegetation. We do this using the `updateMask` function, and specify that we want to remove areas that have an NDVI value higher than 0.2 and and NDWI value higher than 0.3. You can play around with these thesholds until you achieve the desired results.
|
||||
|
||||
Now we can use these indices to filter out water and vegetation. We do this using the `updateMask` function, and specify that we want to remove areas that have an NDVI value higher than 0.2 and anNDWI value higher than 0.3. You can play around with these thresholds until you achieve the desired results.
|
||||
|
||||
|
||||
```js
|
||||
|
||||
|
||||
var image=sentinel.updateMask(ndwi.lt(0.3))
|
||||
.updateMask(ndvi.lt(0.2))
|
||||
.addBands(ndvi)
|
||||
@@ -90,39 +120,53 @@ var image=sentinel.updateMask(ndwi.lt(0.3))
|
||||
```
|
||||
We also want to only select bands that are relevant to our analysis; Sentinel
|
||||
|
||||
|
||||
Finally, let's clip the image to our Area of Interest (AOI) and add it to the map using the visualization parameters we defined earlier.
|
||||
|
||||
|
||||
```js
|
||||
Map.addLayer(image.clip(AOI), s_rgb, 'Sentinel');
|
||||
```
|
||||
|
||||
|
||||
Now that we've loaded and preporcessed our satellite imagery, we can proceed with the rest of our task. Ultimately, we want to create a map of the study area which shows us different "landcovers" (materials). This can broadly be achieved in three steps:
|
||||
|
||||
Now that we've loaded and pre-porcessed our satellite imagery, we can proceed with the rest of our task. Ultimately, we want to create a map of the study area which shows us different "landcovers" (materials). This can broadly be achieved in three steps:
|
||||
|
||||
|
||||
1. Generate labeled landcover data
|
||||
2. Train a model using labeled data
|
||||
3. Validate the model
|
||||
|
||||
|
||||
## 1. Generating Labeled Data
|
||||
|
||||
A vital step in any machine learning workflow is the generation of labeled data, which we will use to train a model to differentiated between different types of land cover and later to test the model's accuracy. By looking around the study area, we can get a sense of the different land cover classes that we might encounter:
|
||||
|
||||
A vital step in any machine learning workflow is the generation of labeled data, which we will use to train a model to differentiate between different types of land cover and later to test the model's accuracy. By looking around the study area, we can get a sense of the different land cover classes that we might encounter:
|
||||
|
||||
|
||||
1. Agricultural Land
|
||||
2. Urban Areas
|
||||
3. Oil Contamination
|
||||
|
||||
|
||||
Naturally we could subdivide each of these into sub-categories, and there are probably other classes we haven't included that may be present in the study area. The choice of classes is partly informed by the nature of the task at hand. In theory, the most efficient number of classes for this task would be two: oil, and everything else. The problem is that the "everything else" category would be pretty noisy since it would include a wide range of materials, making it harder to distinguish this from oil. In practice, a visual inspection of major landcover classes in the study area is a quick-and-dirty way of getting at roughly the right number of classes. This is also an iterative process: you can start with a set of labeled data, look at the model results, and adjust your sampling accordingly. More on this later.
|
||||
|
||||
|
||||
The main landcover class we're interested in is, of course, oil. Some oil contamination is readily visible from the high resolution satellite basemap; rivers of oil flow from the leaking [Ger Zero refinery](https://zoom.earth/#view=36.947921,42.02871,16z/overlays=heat,labels:off,crosshair). We can draw polygons around the oil contamination like so:
|
||||
|
||||
|
||||
|
||||
|
||||
The same process is applied to agricultural land and urban areas. In general, you want to make sure that you're sampling from all across the study area. I've generated between 4-10 polygons per landcover class in different places. We're now left with a featureCollection composed of polygons for each class. I've named them `oil`, `agriculture`, and `urban`.
|
||||
|
||||
The same process is applied to agricultural land and urban areas. In general, you want to make sure that you're sampling from all across the study area. I've generated between four to 10 polygons per landcover class in different places. We're now left with a featureCollection composed of polygons for each class. I've named them `oil`, `agriculture`, and `urban`.
|
||||
|
||||
|
||||
However, I don't just want to use all of the pixels contained in these polygons for training. There are several reasons for this. First, it would likely lead to [overfitting](https://en.wikipedia.org/wiki/Overfitting). Second, there are probably over a million pixels between all of the polygons, which would slow things down unnecessarily. Third, I haven't drawn the polygons to be equal sizes across classes, so I could end up with way more points from one class compared to another. It's OK to have some imbalance between classes, but you don't want it to be extreme.
|
||||
|
||||
|
||||
As such, the next step involves taking random samples of points from *within* these polygons. I do so using the `randomPoints` function:
|
||||
|
||||
|
||||
```js
|
||||
var oil_points=ee.FeatureCollection.randomPoints(oil, 3000).map(function(i){
|
||||
return i.set({'class': 0})})
|
||||
@@ -134,10 +178,13 @@ var agriculture_points=ee.FeatureCollection.randomPoints(agriculture, 2000).map(
|
||||
return i.set({'class': 2})})
|
||||
```
|
||||
|
||||
|
||||
In the first line, I create a new featureCollection called `oil_points` which contains 3000 points sampled from the polygons in the `oil` featureCollection. I then map through each of these points, and set a property called "class" equal to 0. I do the same for the urban and agricultural areas, setting the "class" property of these featureCollections to 1 and 2, respectively. Ultimately, our model will output a raster in which each pixel will contain one of these three values. A value of 0 in the output will represent the model predicting that that pixel is oil, based on the training data; a value of 1 would indicate predicted urban land cover, and 2 predicted agricultural landcover.
|
||||
|
||||
|
||||
Now we want to create one feature collection called "sample", which will contain all three sets of points.
|
||||
|
||||
|
||||
```js
|
||||
var sample=ee.FeatureCollection([oil_points,
|
||||
urban_points,
|
||||
@@ -147,19 +194,25 @@ var sample=ee.FeatureCollection([oil_points,
|
||||
.randomColumn();
|
||||
```
|
||||
|
||||
|
||||
We've also assigned a property called "random" using the `randomColumn` function. This lets us split our featureCollection into two: one used for training the model, and one used for validation. We'll use a 70-30 split.
|
||||
|
||||
|
||||
```js
|
||||
var split=0.7
|
||||
var training_sample = sample.filter(ee.Filter.lt('random', split));
|
||||
var validation_sample = sample.filter(ee.Filter.gte('random', split));
|
||||
```
|
||||
|
||||
|
||||
## 2. Training a Model
|
||||
|
||||
|
||||
Having generated labeled training and testing data, we now want to teach an algorithm to associate the pixels in those areas (in particular, their spectral profiles) with a specific landcover class.
|
||||
|
||||
The list of points we generated in the previous step contain a label (0: oil, 1: urban, 2: agriculture). However, they do not yet contain any information about the spectral profile of the Sentinel-2 image. The `sampleRegions` function lets us assign a the band values from an image as properties to our feature collection. We do this for both training sample and the validation sample.
|
||||
|
||||
The list of points we generated in the previous step contain a label (0: oil, 1: urban, 2: agriculture). However, they do not yet contain any information about the spectral profile of the Sentinel-2 image. The `sampleRegions` function lets us assign the band values from an image as properties to our feature collection. We do this for both the training sample and the validation sample.
|
||||
|
||||
|
||||
```js
|
||||
var training = image.sampleRegions({
|
||||
@@ -168,6 +221,7 @@ var training = image.sampleRegions({
|
||||
scale: 10,
|
||||
});
|
||||
|
||||
|
||||
var validation = image.sampleRegions({
|
||||
collection: validation_sample,
|
||||
properties: ['class'],
|
||||
@@ -175,53 +229,70 @@ var validation = image.sampleRegions({
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
Each point in the featureCollections above will contain a property denoting each Sentinel-2 band's value at that location, as well as the property denoting the class label.
|
||||
|
||||
|
||||
Now we're ready to train the model. We'll be using a [Random Forest](https://en.wikipedia.org/wiki/Random_forest) classifier, which basically works by trying to separate your data into the specified classes by setting lots of thresholds in your input properties (in our case, Sentinel-2 band values). It's a versatile and widely-used model.
|
||||
|
||||
|
||||
We first call a random forest classifier with 500 trees. More trees usually yields higher accuracy, though there are diminishing returns. Too many trees will result in your computation timing out. We then train the model using the `train` function, which we supply with the training data as well as the name of the property that contains our class labels ("class").
|
||||
|
||||
|
||||
```js
|
||||
var model = ee.Classifier.smileRandomForest(500)
|
||||
.train(training, 'class');
|
||||
```
|
||||
|
||||
The trained model now associates Sentinel-2 band values with one of three landcover classes. We can now feed the model pixels it has never seen before, and it will use what it now knows about the spectral profiles of the differnt classes to predict the class of the new pixel.
|
||||
|
||||
The trained model now associates Sentinel-2 band values with one of three landcover classes. We can now feed the model pixels it has never seen before, and it will use what it now knows about the spectral profiles of the different classes to predict the class of the new pixel.
|
||||
|
||||
|
||||
```js
|
||||
var prediction = image.classify(model)
|
||||
```
|
||||
`prediction` is now a raster which contains one of three values (0: oil, 1: urban, 2: agriculture). We're only interested in oil, so let's isolate the regions in this raster that have a value of 0, and add them in red to the map:
|
||||
|
||||
|
||||
```js
|
||||
var oil_prediction=prediction.updateMask(prediction.eq(0))
|
||||
|
||||
|
||||
Map.addLayer(oil_prediction, {palette:'red'}, 'Predicted Oil Conamination')
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 3. Validation
|
||||
|
||||
The image above should look somewhat familiar. It's Ger Zero, where we trained part of our model. We can see in red the areas which the model predicts to be oil pollution. These largley align with the areas that we can see as being contaminated based on the high resolution basemap. It's not perfect, but it's pretty good.
|
||||
|
||||
The image above should look somewhat familiar. It's Ger Zero, where we trained part of our model. We can see in red the areas which the model predicts to be oil pollution. These largely align with the areas that we can see as being contaminated based on the high resolution basemap. It's not perfect, but it's pretty good.
|
||||
|
||||
|
||||
Let's scroll to another area, far from where the model was trained.
|
||||
|
||||
This image shows two clusters of makeshift refineries which were identified by the model. This is good, though we can only get so far by visually inspecting the output from our model. To get a better sense of our model's performance, we can use the validation data that we generated previously. Remember, these are labeled points which our model was not trained on, and has never seen before.
|
||||
|
||||
|
||||
We'll take the `validation` featureCollection containing our labeled points, and have our model classify it.
|
||||
|
||||
|
||||
```js
|
||||
var validated = validation.classify(model);
|
||||
```
|
||||
|
||||
|
||||
Now the `validated` variable is a featureCollection which contains both manual labels and predicted labels from our model. We can compare the manual labels to the predicted output to get a sense of how well our model is performing. This is called a Confusion Matrix (or an Error Matrix):
|
||||
|
||||
|
||||
```js
|
||||
var testAccuracy = validated.errorMatrix('class', 'classification');
|
||||
|
||||
|
||||
print('Confusion Matrix ', testAccuracy);
|
||||
```
|
||||
|
||||
|
||||
| | | | *Labels* | |
|
||||
|:------------:|:---------------:|:-------:|:---------:|:---------------:|
|
||||
| | | **Oil** | **Urban** | **Agriculture** |
|
||||
@@ -229,25 +300,34 @@ print('Confusion Matrix ', testAccuracy);
|
||||
| *Prediction* | **Urban** | 0 | 168 | 8 |
|
||||
| | **Agriculture** | 1 | 4 | 514 |
|
||||
|
||||
Now, we can see that of the 877 points that were labeled "oil", only one was falsely predicted to be agicultural land. The model also falsely predicted as oil one point that was labeled urban, and five points that were labeled agriculture. Not bad. We can get a sense of the model's overall accuracy using the `accuracy` function on the confusion matrix:
|
||||
|
||||
Now, we can see that of the 877 points that were labeled "oil", only one was falsely predicted to be agricultural land. The model also falsely predicted as oil one point that was labeled urban, and five points that were labeled agriculture. Not bad. We can get a sense of the model's overall accuracy using the `accuracy` function on the confusion matrix:
|
||||
|
||||
|
||||
```js
|
||||
print('Validation overall accuracy: ', testAccuracy.accuracy())
|
||||
```
|
||||
This tells us that the overall accuracy of our model is around 98%. However, we shouldn't take this estimate at face value. There are a number of complicated reasons ([spatial autocorrelation](https://www.sciencedirect.com/topics/computer-science/spatial-autocorrelation#:~:text=Spatial%20autocorrelation%20is%20the%20term,together%20to%20have%20similar%20values.) in the training data, for example) why this figure is probably inflatred. If we were submitting this analysis to a peer-reviewed journal, we'd take great care in addressing this, but for our purposes we can use the accuracy statistics to guide our analysis and get a rough sense of how well the model is performing.
|
||||
This tells us that the overall accuracy of our model is around 98%. However, we shouldn't take this estimate at face value. There are a number of complicated reasons ([spatial autocorrelation](https://www.sciencedirect.com/topics/computer-science/spatial-autocorrelation#:~:text=Spatial%20autocorrelation%20is%20the%20term,together%20to%20have%20similar%20values.) in the training data, for example) why this figure is probably inflated. If we were submitting this analysis to a peer-reviewed journal, we'd take great care in addressing this, but for our purposes we can use the accuracy statistics to guide our analysis and get a rough sense of how well the model is performing.
|
||||
|
||||
|
||||
This model isn't perfect; it often mis-classifies the shorelines of lakes as oil, or certain parts of urban areas. As previously mentioned, training a model is often an iterative process. At this stage, if your accuracy is not as high as you'd like it to be, you can use the output to figure out how to tweak the model. For example, you may observe that your model is confusing urban areas with oil spills. You can draw a polygon over the erroneous area, label it urban landcover and retrain the model thereby hopefully improving accuracy. We could further refine our model in this way.
|
||||
|
||||
This model isn't perfect; it often misclassifies the shorelines of lakes as oil, or certain parts of urban areas. As previously mentioned, training a model is often an iterative process. At this stage, if your accuracy is not as high as you'd like it to be, you can use the output to figure out how to tweak the model. For example, you may observe that your model is confusing urban areas with oil spills. You can draw a polygon over the erroneous area, label it urban landcover and retrain the model thereby hopefully improving accuracy. We could further refine our model in this way.
|
||||
|
||||
# Communicating the Results
|
||||
|
||||
|
||||
Now that we've got a model that can identify oil from multispectral satellite imagery fairly well, we can set about making our results accessible.
|
||||
|
||||
|
||||
One of the things we're particularly interested in is the distribution of small refineries. The way we're currently visualizing the prediction (the raster output from the model where predicted oil is shown in red and everything else is transparent) makes it hard to see these small refineries when we zoom out:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
We can convert our raster into a series of points using the `reduceToVectors` function. In essence, this takes homogenous regions of an image (e.g., an area predicted to be oil surrounded by an area not predicted to be oil) and converts it into a point:
|
||||
|
||||
|
||||
```js
|
||||
var vectors = oil_prediction.reduceToVectors({
|
||||
geometry: AOI,
|
||||
@@ -258,15 +338,18 @@ var vectors = oil_prediction.reduceToVectors({
|
||||
maxPixels:1653602926
|
||||
}).filterBounds(AOI)
|
||||
|
||||
|
||||
Map.addLayer(vectors.style({color: 'black', fillColor: '#00f2ff', pointSize:5}),{},'Oil Contamination Points',false)
|
||||
```
|
||||
|
||||
|
||||
Now the distribution of small refineries is much more easily visible as blue dots:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
If we zoom out even further, we can see clusters of points that correspond to areas of high oil production. Using geolocated photographs, we can roughly ground-truth the model output:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
# Blast Damage Assessment {.unnumbered}
|
||||
---
|
||||
title: Blast Damage Assessment
|
||||
---
|
||||
|
||||
On August 4th, 2020, a warehouse full of fertilizer in the port of Beirut exploded. The blast killed over 200 people, injured thousands, and caused widespread damage to the city:
|
||||
|
||||
|
||||
|
||||
|
||||
<div style="padding-bottom:56.25%; position:relative; display:block; width: 100%">
|
||||
<iframe width="100%" height="100%"
|
||||
src="https://www.youtube.com/embed/LNDhIGR-83w?start=29"
|
||||
@@ -11,63 +15,90 @@ On August 4th, 2020, a warehouse full of fertilizer in the port of Beirut explod
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
|
||||
Assessing blast damage is a common task for open source investigators, and satellite imagery analysis is one of the best tools we have at our disposal to analyze this sort of phenomenon. [NASA](https://earthobservatory.nasa.gov/images/147098/scientists-map-beirut-blast-damage) used Sentinel-1 imagery in the aftermath of the explosion to generate an estimated damage map. They explain that Sentinel-1 Synthetic Aperture Radar (SAR) imagery is good for this sort of task:
|
||||
|
||||
|
||||
"SAR instruments send pulses of microwaves toward Earth’s surface and listen for the reflections of those waves. The radar waves can penetrate cloud cover, vegetation, and the dark of night to detect changes that might not show up in visible light imagery. When Earth’s crust moves due to an earthquake, when dry land is suddenly covered by flood water, or when buildings have been damaged or toppled, the amplitude and phase of radar wave reflections changes in those areas and indicates to the satellite that something on the ground has changed."
|
||||
|
||||
The NASA team produced this estimate the day after the explosion, which is very impressive. However, due to the quick turnaround, were pretty light on the description of their methodology (they didnt provide any code, or even explicity say how exactly they generated the estimate), making their analysis hard to replicate. They also failed to validate their results, which is a critical step in any analysis.
|
||||
|
||||
The NASA team produced this estimate the day after the explosion, which is very impressive. However, due to the quick turnaround, they were pretty light on the description of their methodology (they didn’t provide any code, or even explicitly say how exactly they generated the estimate), making their analysis hard to replicate. They also failed to validate their results, which is a critical step in any analysis.
|
||||
|
||||
|
||||
In this case study we'll be developing our own change detection algorithm from scratch, applying to Sentinel-1 imagery of Beirut before and after the blast, and validating our results using building footprints and U.N. damage estimates as the ground truth. Below is the final result of the analysis, which shows building footprints colored according to the predicted level of damage they sustained from the blast:
|
||||
|
||||
|
||||
:::{.column-page}
|
||||
|
||||
|
||||
<iframe src='https://ollielballinger.users.earthengine.app/view/beirutsar' width='100%' height='700px'></iframe>
|
||||
|
||||
|
||||
:::
|
||||
|
||||
|
||||
|
||||
|
||||
## Change Detection
|
||||
|
||||
|
||||
There are many ways to detect change between two images. The simplest way would be to take an image taken before an event, and subtract it from an image taken after. This is a good way to get a general sense of where change has occurred, but if you only use two images (one before an event and another after), it would be difficult to differentiate between areas that have changed as a result of the event in question, and areas that have changed for other reasons. Things in Beirut (and cities in general) are constantly changing: construction, cars/planes/ships moving, vegetation growing, etc. So we wouldn't know if the change we're seeing is a result of the explosion or whether that area is generally prone to change. We can overcome this by comparing a bunch of pre-event images to a bunch of post-event images. This way we can see if the change we're seeing is consistent across all of the images. If it is, then we can be fairly confident that the change is a result of the event in question. The mean is simply the sum of all the values ($x_i$) in a set divided by the number of values ($n$):
|
||||
|
||||
|
||||
$$\large \overline{x} = \frac{1}{n} \sum_{i=1}^n x_i$$
|
||||
|
||||
|
||||
But if we just take the average pixel value before and subtract the average pixel value after, we're not accounting for the *variability* of that pixel's values. For example, if we have a pixel that has had an average value of 1 for the month before the event, and a value of 2 in the month after the event, the difference is 1. If that pixel's value is extremely consistent (it never varies by more than 0.1), such a change would be very significant. But if that pixel's value is very variable (it varies by 2 or even 3 on a regular basis), then the change is not significant. So we need to account for the variability of the pixel's values using the standard deviation. It is calculated as the square root of the variance, which is the average of the squared differences from the mean:
|
||||
|
||||
|
||||
$$\large s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$
|
||||
|
||||
|
||||
With just the mean and the standard deviations of two sets of numbers, we can use a statistical test to determine whether the change in means is significant. The simplest way to do this is to use a pixelwise t-test, which is basically just a signal-to-noise ratio: it calculates the difference between two sample means (signal), and divides it by the standard deviations of both samples (noise). In this case, the two samples are the pre- and post-event images. The t-test is applied to each pixel in the image, allowing us to determine whether the change is statistically significant. Given two groups, $x_1$ before the event, and $x_2$ after the event, the $t$ statistic is calculated as:
|
||||
|
||||
|
||||
$$ \Large t = {\frac{\overline{x_1}-\overline{x_2}} {\sqrt{\frac{s^2_1}{n_1} + \frac{s^2_2}{n_2}}}} $$
|
||||
|
||||
|
||||
Where:
|
||||
|
||||
|
||||
* $\overline{x}$: Sample Mean
|
||||
* $s^2$: Sample Standard Deviation
|
||||
* $n$: Number of observations
|
||||
|
||||
|
||||
This procedure gives us a number called a t-value, which is a measure of how many standard deviations the difference between the two means is. We're not going to get into the details here, but a rule of thumb is that if the t-value is greater than 2, then the difference between the two means is significant. If the t-value is less than 2, then the difference is not significant. We're going to calculate the t-value for each pixel in the image to determine whether that pixel has changed significantly following the event in question. You don't need to know the details of the t-test to understand the results (but hopefully you've got an intuition for what it's doing). If you're interested in learning more about statistical tests of this sort, I teach a course on Data Science at the University College London, and have made all of the lectures and courseware open-source. The T-test lecture is here.
|
||||
|
||||
|
||||
## Implementing a t-test in Earth Engine
|
||||
|
||||
|
||||
Now lets go about implementing this in Earth Engine. We'll start by centering the map on the port of Beirut, and setting the map to satellite view, and defining an area of interest (AOI) as a 3km buffer around the port:
|
||||
|
||||
|
||||
```js
|
||||
Map.setCenter(35.51898, 33.90153, 15);
|
||||
|
||||
|
||||
Map.setOptions("satellite");
|
||||
|
||||
|
||||
var aoi = ee.Geometry.Point(35.51898, 33.90153).buffer(3000);
|
||||
```
|
||||
Next, let's define a function in earth engine that will perform the T-Test. The block of code below defines a function to implement a t-test for every pixel in a set of images. The function will be called 'ttest', and takes four arguments:
|
||||
|
||||
|
||||
* s1: the image collection
|
||||
* shock: the date of the event
|
||||
* pre_interval: the number of months before the event
|
||||
* post_interval: the number of months after the event
|
||||
|
||||
|
||||
The function will return a t-value image, which we can use to determine whether a pixel has changed significantly following the event in question.
|
||||
|
||||
|
||||
```js
|
||||
function ttest(s1, shock, pre_interval, post_interval) {
|
||||
|
||||
@@ -99,16 +130,20 @@ function ttest(s1, shock, pre_interval, post_interval) {
|
||||
.divide(pre_n.add(post_n).subtract(2))
|
||||
.sqrt();
|
||||
|
||||
|
||||
// Calculate the denominator of the t-test
|
||||
var denom = pooled_sd.multiply(
|
||||
ee.Number(1).divide(pre_n).add(ee.Number(1).divide(post_n)).sqrt()
|
||||
);
|
||||
|
||||
|
||||
// Calculate the Degrees of Freedom, which is the number of observations minus 2
|
||||
var df = pre_n.add(post_n).subtract(2);
|
||||
|
||||
|
||||
print("Number of Images: ", df);
|
||||
|
||||
|
||||
// Calculate the t-test using the:
|
||||
// mean of the pre-event period,
|
||||
// the mean of the post-event period,
|
||||
@@ -120,20 +155,26 @@ function ttest(s1, shock, pre_interval, post_interval) {
|
||||
.abs()
|
||||
.subtract(2);
|
||||
|
||||
|
||||
// return the t-values for each pixel
|
||||
return change;
|
||||
}
|
||||
```
|
||||
An important detail in the code above is that we've actually tweaked the t-test slightly, in two ways.
|
||||
|
||||
|
||||
First, the algorithm above returns tha *absolute* value of t (i.e. the absolute value of the difference between the two means). This is because we're interested in whether the pixel has changed at all, not whether it's changed in a particular direction. Second, we've subtracted 2 from the t-value.
|
||||
|
||||
The t-value is a measure of how many standard deviations the difference between the two means is. Generally speaking, if the t-value is greater than 2, then the difference between the two means is considered statistically significant. 2 is a fairly abitrary cutoff, but it's the most commonly used one since it corresponds to the 95% confidence interval (i.e., theres less than a 5% chance of observing a difference that big due to random chance). Now we've got a function that can take an image collection and return a t-value image, where a value greater than 0 corresponds to a statistically significant change between the pre-event and post-event periods.
|
||||
|
||||
The t-value is a measure of how many standard deviations the difference between the two means is. Generally speaking, if the t-value is greater than 2, then the difference between the two means is considered statistically significant. 2 is a fairly arbitrary cutoff, but it's the most commonly used one since it corresponds to the 95% confidence interval (i.e., there’s less than a 5% chance of observing a difference that big due to random chance). Now we've got a function that can take an image collection and return a t-value image, where a value greater than 0 corresponds to a statistically significant change between the pre-event and post-event periods.
|
||||
|
||||
|
||||
## Filtering the Sentinel-1 Imagery
|
||||
|
||||
|
||||
We can't just blindly apply this algorithm to the entire image collection, because the image collection contains images from both ascending and descending orbits. We need to filter the image collection to the ascending and descending orbits, and then calculate the t-value for each orbit separately: this is because the satellite is viewing the scene from a completely different angle when it's in ascending and descending orbits, which will generate a lot of noise in our data. In fact, even when the satellite is either ascending or descending, we can have multiple images of the same place taken from slightly different orbital tracks because these overlap (see [this visualization of orbits](/ch1#orbits)). We need to filter the image collection to the relative orbit number that is most common within the image collection. For that, we define a new function called 'filter_s1', which takes a single argument: the path (either 'ASCENDING' or 'DESCENDING').
|
||||
|
||||
|
||||
```js
|
||||
function filter_s1(path) {
|
||||
|
||||
@@ -146,41 +187,51 @@ function filter_s1(path) {
|
||||
.filterBounds(aoi)
|
||||
.select("VH");
|
||||
|
||||
|
||||
// Find the most common relative orbit number
|
||||
var orbit = s1
|
||||
.aggregate_array("relativeOrbitNumber_start")
|
||||
.reduce(ee.Reducer.mode());
|
||||
|
||||
|
||||
// Filter the image collection to the most common relative orbit number
|
||||
var s1 = s1.filter(ee.Filter.eq("relativeOrbitNumber_start", orbit));
|
||||
|
||||
|
||||
// Calculate the t-test for the filtered image collection using the function we defined earlier
|
||||
var change = ttest(s1, "2020-08-04", 12, 2);
|
||||
|
||||
|
||||
// Return the t-values for each pixel
|
||||
return change;
|
||||
}
|
||||
```
|
||||
You'll notice that we've called the ttest function we defined earlier with four arguments:
|
||||
|
||||
|
||||
* s1: the Sentinel-1 image collection filtered to the ascending or descending orbit
|
||||
* 2020-08-04: the date of the explosion
|
||||
* 12: the number of months before the explosion to use for the pre-event period; I'm choosing to include the full year prior to the explosion to get a good baseline
|
||||
* 2: the number of months after the explosion to use for the post-event period; I'm including 2 months after the explosion. Much less than that, and we risk missing the effects of the explosion. Much more than that, and we risk including the effects other changes that happened after the explosion, including the reconstruction effort.
|
||||
|
||||
|
||||
Now we want to apply this function to the image collection twice (once for each orbit) and then combine the two images into a single image. After that, we can clip it to the area of interest and display it on the map:
|
||||
|
||||
|
||||
```js
|
||||
|
||||
|
||||
// Call the filter_s1 function twice, once for each orbit, and then combine the two images into a single image
|
||||
var composite = ee
|
||||
.ImageCollection([filter_s1("ASCENDING"), filter_s1("DESCENDING")])
|
||||
.mean()
|
||||
.clip(aoi);
|
||||
|
||||
|
||||
// Define a color palette
|
||||
var palette = ["440154", "3b528b", "21918c", "5ec962", "fde725"];
|
||||
|
||||
|
||||
// Add the composite to the map
|
||||
Map.addLayer(
|
||||
composite,
|
||||
@@ -188,35 +239,47 @@ Map.addLayer(
|
||||
"change"
|
||||
);
|
||||
```
|
||||
The visualization parameters correspond the statitical significance of the change in pixel values. Using the Viridis color palette which ranges from purple to yellow, dark purple pixels indicate no significant change, and yellow pixels indicate a significant change with with 95% confidence. The brighter the yellow of a pixel, the more significant the change.
|
||||
The visualization parameters correspond to the statistical significance of the change in pixel values. Using the Viridis color palette which ranges from purple to yellow, dark purple pixels indicate no significant change, and yellow pixels indicate a significant change with with 95% confidence. The brighter the yellow of a pixel, the more significant the change.
|
||||
|
||||
|
||||
|
||||
This seems to be working quite well; but remember, ports are generally prone to change. The t-test is accounting for this by calculating each pixel's variance over the entire time period, but it's still possible that the change we're seeing is due to the port rather than the explosion. To test this, we can run the same algorithm on the same area, using the same date cutoff (August 4th), but in a different year; i've chosen 2018. This is what's known as a placebo test: if it's still showing loads of statistically significant change around the cutoff, our algorithm is probably picking up on port activity rather than the explosion.
|
||||
|
||||
|
||||
|
||||
|
||||
Compared to the 2020 image, there's a lot less yellow (significant change). That being said there are a few yellow areas. This could be due to a number of reasons: ships coming and going, cranes moving, and containers being loaded and unloaded would all register in the change detection algorithm. There are also a number of yellow specks throughout the city, which is also to be expected since cities are also generally in a state of flux. Construction, demolition, and even the growth of vegetation can all be detected by the algorithm.
|
||||
This seems to be working quite well; but remember, ports are generally prone to change. The t-test is accounting for this by calculating each pixel's variance over the entire time period, but it's still possible that the change we're seeing is due to the port rather than the explosion. To test this, we can run the same algorithm on the same area, using the same date cutoff (August 4th), but in a different year; I've chosen 2018. This is what's known as a placebo test: if it's still showing loads of statistically significant change around the cutoff, our algorithm is probably picking up on port activity rather than the explosion.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Compared to the 2020 image, there's a lot less yellow (significant change). That being said, there are a few yellow areas. This could be due to a number of reasons: ships coming and going, cranes moving, and containers being loaded and unloaded would all register in the change detection algorithm. There are also a number of yellow specks throughout the city, which is also to be expected since cities are also generally in a state of flux. Construction, demolition, and even the growth of vegetation can all be detected by the algorithm.
|
||||
|
||||
|
||||
However, the scale and quantity of the change is nowhere near what it was for the 2020 image. This is a good sign that the algorithm is detecting change resulting from the explosion.
|
||||
|
||||
However, the scale and quantity of the change is nowhere near what it was for the 2020 image. This is a good sign that the algorithm detecting change resulting from the explosion.
|
||||
|
||||
## Validation
|
||||
|
||||
Great. We've developed our very own change detection algorithm in earth engine, applied it to the Beirut explosion, and it seems to be working using a basic placebo test. But how do we know that it's correctly predicting the *extent* of the damage, and not wildly over/underestimating?
|
||||
|
||||
Given that this was a few years ago, we have the benefit of hindsight. In particular, the United Nations and the Municipality of Beirut have [published a report](https://unhabitat.org/sites/default/files/2020/10/municipality_of_beirut_-_beirut_explosion_rapid_assessment_report.pdf) on the damage caused by the explosion. This report includes estimates of the number of buildings damaged or destroyed by the explosion, as well as the number of people displaced. The report states that approximately 10,000 buildings were damaged within a 3km radius of the port. If our algorithm suggests that only 1,000 buildings were damaged, it's undershooting. If it suggests that 100,000 buildings were damaged, it's overshooting.
|
||||
Great. We've developed our very own change detection algorithm in earth engine, applied it to the Beirut explosion, and it seems to be working after checking with a basic placebo test. But how do we know that it's correctly predicting the *extent* of the damage, and not wildly over/underestimating?
|
||||
|
||||
|
||||
Given that this was a few years ago, we have the benefit of hindsight. In particular, the United Nations and the Municipality of Beirut have [published a report](https://unhabitat.org/sites/default/files/2020/10/municipality_of_beirut_-_beirut_explosion_rapid_assessment_report.pdf) on the damage caused by the explosion. This report includes estimates of the number of buildings damaged or destroyed, as well as the number of people displaced. The report states that approximately 10,000 buildings were damaged within a 3 kilometre radius of the port. If our algorithm suggests that only 1,000 buildings were damaged, it's undershooting. If it suggests that 100,000 buildings were damaged, it's overshooting.
|
||||
|
||||
|
||||
Using building footprint data and the t-test image we just generated, we can createe an estimate of the number of damaged buildings according to our model. First, we want to generate a thresholded image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0. We can then use this mask to reduce the building footprints to a single value for each building, where the value is the mean of the t-test image within the footprint. If the mean value is greater than 0, the building is damaged. If it's less than 0, the building is not damaged.
|
||||
|
||||
Using building footprint data and the t-test image we just generated, we can generate an estimate of the number of damaged buildings according to our model. First, we want to generate a thresholded image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0. We can then use this mask to reduce the building footprints to a single value for each building, where the value is the mean of the t-test image within the footprint. If the mean value is greater than 0, the building is damaged. If it's less than 0, the building is not damaged.
|
||||
|
||||
```js
|
||||
// Create a mask of the t-test image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0
|
||||
var threshold = composite.updateMask(composite.gt(0));
|
||||
|
||||
|
||||
// Load the building footprints
|
||||
var buildings = ee
|
||||
.FeatureCollection("projects/sat-io/open-datasets/MSBuildings/Lebanon")
|
||||
.filterBounds(aoi);
|
||||
|
||||
|
||||
// Calculate the mean value of the t-test image within each building footprint
|
||||
var damaged_buildings = threshold.reduceRegions({
|
||||
collection: buildings,
|
||||
@@ -224,24 +287,31 @@ var damaged_buildings = threshold.reduceRegions({
|
||||
scale: 1,
|
||||
});
|
||||
|
||||
|
||||
// Print the number of buildings with a mean value greater than 0
|
||||
// i.e., those displaying statistically significant change
|
||||
print(damaged_buildings.filter(ee.Filter.gt("mean", 0)).size());
|
||||
```
|
||||
|
||||
|
||||
The result is 9,256, which is pretty damn close to 10,000. We can also visualize the building footprints on the map, colored according the mean value of the t-test image within the footprint, where:
|
||||
|
||||
|
||||
* Blue = no damage
|
||||
* Green = low damage
|
||||
* Yellow/Orange = medium damage
|
||||
* Red = high levels of damage
|
||||
|
||||
|
||||
|
||||
|
||||
```js
|
||||
|
||||
|
||||
// Create an empty image
|
||||
var empty = ee.Image().byte();
|
||||
|
||||
|
||||
// Paint the building footprints onto the empty image
|
||||
var outline = empty.paint({
|
||||
featureCollection: damaged_buildings,
|
||||
@@ -249,6 +319,7 @@ var outline = empty.paint({
|
||||
width: 5,
|
||||
});
|
||||
|
||||
|
||||
// Define a color palette
|
||||
var building_palette = [
|
||||
"0034f5",
|
||||
@@ -260,6 +331,7 @@ var building_palette = [
|
||||
"fd3000",
|
||||
];
|
||||
|
||||
|
||||
// Add the image to the map
|
||||
Map.addLayer(
|
||||
outline,
|
||||
@@ -268,162 +340,239 @@ Map.addLayer(
|
||||
);
|
||||
```
|
||||
|
||||
The result naturally resembles the underlying t-test image, with high levels of damage concetrated around the port, and progressively decreasing damage with distance:
|
||||
|
||||
|
||||
The result naturally resembles the underlying t-test image, with high levels of damage concentrated around the port, and progressively decreasing damage with distance:
|
||||
|
||||
To get a better sense of how these predicitons correspond to actual damage, we can zoom in and turn on the Google satellite basemap, which has imagery taken just after the explosion; you can still see capsized boats in the port. Zooming in to the epicentre, we can see several warehouses that were effectively vaporized. Our change detection algorithm picks up on a high degree of change, as indicated by the red outlines of the building footprints:
|
||||
|
||||
|
||||
|
||||
|
||||
This is pretty low-hanging fruit. Let's look at a different area, around 1.3km east from the epicentre with a mix of warehouses and residential buildings:
|
||||
|
||||
|
||||
To get a better sense of how these predicitions correspond to actual damage, we can zoom in and turn on the Google satellite basemap, which has imagery taken just after the explosion; you can still see capsized boats in the port. Zooming in to the epicenter, we can see several warehouses that were effectively vaporized. Our change detection algorithm picks up on a high degree of change, as indicated by the red outlines of the building footprints:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
This is pretty low-hanging fruit. Let's look at a different area, around 1.3km east from the epicenter with a mix of warehouses and residential buildings:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Here, there's greater variation in the predictions. I've highlighted three areas.
|
||||
|
||||
In Area A, we see a warehouse with a highily deformed roof; panels of corrugated iron are missing, and much of the roof is warped. The building footprint for this warehouse is red, suggesting that our algorithm correctly predicts a significant amount of blast damage.
|
||||
|
||||
In Area B, we see a medium-rise building. If you look closely at the southern edge of the building, you'll see the siding has been completely torn off and is laying on the sidewalk. The bulding footprint is orange, suggesting a medium amount of change. We may be underestimating a bit here.
|
||||
In Area A, we see a warehouse with a highly deformed roof; panels of corrugated iron are missing, and much of the roof is warped. The building footprint for this warehouse is red, suggesting that our algorithm correctly predicts a significant amount of blast damage.
|
||||
|
||||
|
||||
In Area B, we see a medium-rise building. If you look closely at the southern edge of the building, you'll see the siding has been completely torn off and is laying on the sidewalk. The building footprint is orange, suggesting a medium amount of change. We may be underestimating a bit here.
|
||||
|
||||
|
||||
In Area C, there are a bunch of high rise buildings clustered together. The building footprints are all blue, suggesting little to no damage. This is a bit of a surprise given how damaged areas A and B are. If you squint at the satellite image, it is indeed hard to tell if these buildings are damaged because we're looking at them from the top down, when much of the damage (e.g., the windows being blown out) would only be visible from the side. Indeed, our own estimate of the number of damaged buildings based on the algorithm we developed is about 8% shy of the U.N.'s estimate. This may be why.
|
||||
|
||||
|
||||
## Conclusion
|
||||
|
||||
In this practical, we created a custom change detection algorithm that conducts a pixelwise t-test to detect change resulting from the 2020 explosion in the port of Beirut. By defining our own functions to do most of this analysis, we can apply the same workflow quite easily to a different context by simply moving the AOI and inputting the date of the shock. A placebo test showed that it's not just detecting general change in the area, but specifically change resulting from the explosion: when we keep everythgin the same but change the year of the shock, we see very little significant change being detected. Finally, by joining the predicted damage map to building footprints, we come up with an estimate of 9,256 damaged buildings, which is pretty close to the U.N.'s estimate of 10,000. That concludes the portion of this case study that deals with Earth Engine, but if you're interested in learning more about why we're coming up a bit short on the damage estimate (and some different ways of looking at the problem), read on.
|
||||
|
||||
In this practical example, we created a custom change detection algorithm that conducts a pixelwise t-test to detect change resulting from the 2020 explosion in the port of Beirut. By defining our own functions to do most of this analysis, we can apply the same workflow quite easily to a different context by simply moving the AOI and inputting the date of the shock. A placebo test showed that it's not just detecting general change in the area, but specifically change resulting from the explosion: when we keep everything the same but change the year of the shock, we see very little significant change being detected. Finally, by joining the predicted damage map to building footprints, we come up with an estimate of 9,256 damaged buildings, which is pretty close to the U.N.'s estimate of 10,000. That concludes the portion of this case study that deals with Earth Engine, but if you're interested in learning more about why we're coming up a bit short on the damage estimate (and some different ways of looking at the problem), read on.
|
||||
|
||||
|
||||
## Extension: Satellite Imagery and its Limits
|
||||
|
||||
|
||||
Though satellite imagery analysis is undoubtedly one of the best tools we have at our disposal to analyze this sort of phenomenon, it appears to systematically underestimate the extent of damage in Beirut. I outline an alternative approach using Open Street Map data to create a 3D model of Beirut and the explosion to analyze directional blast damage. Again, we're now leaving Earth Engine and moving to Blender, so if you're not interested in that feel free to skip ahead to the next case study.
|
||||
|
||||
|
||||
Below is one of the most viewed videos of the explosion:
|
||||
|
||||
|
||||
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Stunning video shows explosions just minutes ago at Beirut port <a href="https://t.co/ZjltF0VcTr">pic.twitter.com/ZjltF0VcTr</a></p>— Borzou Daragahi 🖊🗒 (@borzou) <a href="https://twitter.com/borzou/status/1290675854767513600?ref_src=twsrc%5Etfw">August 4, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
|
||||
|
||||
|
||||
Geolocating this video was pretty simple thanks to the Greek Orthodox church (highlighted in green below) and the road leading to it (highlighted in blue). The red box indicates the likely location (33.889061, 35.515909) from which the person was filming:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The video shows heavy damage being sustained by areas well outside the zones classified as damaged in the maps above (both my own and NASA's). Indeed, substantial damage was reported several kilometers away.
|
||||
|
||||
|
||||
Why are satellite images underestimating damage in Beirut? Satellite images are taken from above, and are two-dimensional. Much of the damage caused by the blast, however, was directional; the pressure wave hit the sides of buildings, as shown in this diagram from a FEMA manual:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Areas close to the explosion suffered so much damage that it could be seen from above, but even if an apartment building had all of its windows blown out, this would not necessarily be visible in a top-down view. Even for radar, which does technically collect data in three dimensions, the angle problem remains; a high resolution radar might be able to tell you how tall an apartment complex is, but it won't give you a clear image of all sides. Case in point: the NASA damage map was created using Sentinel-1 SAR data. In a nutshell, damage assessment in this case is a three-dimensional problem, and remote sensing is a two-dimensional solution.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Creating a 3D model of Beirut
|
||||
|
||||
|
||||
To create a more accurate rendering of directional blast damage, three dimensional data are required. Data from Open Street Maps (OSM) contains information on both the "footprints" (i.e., the location and shape) as well as the height of buildings, which is enough to create a three dimensional model of Beirut. 3D rendering was done in Blender using the Blender-OSM add-on to import a satellite basemap, terrain raster, and OSM data.
|
||||
|
||||
|
||||
Geolocated videos of the blast can be used to verify and adjust the model. Below is a side-by-side comparison of the twitter video and a 3D rendition of OSM data:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Some slight adjustments to the raw OSM data were made to achieve the image on the right. The building footprints are generally very accurate and comprehensive in coverage, but the building height data does occasionally have to be adjusted manually. A simple and reliable way of doing this is to look at the shadows cast by the building on the satellite base map and scale accordingly. I also added a rough texture to the buildings to help differentiate them, and added the domed roof of the Greek Orthodox church for reference.
|
||||
|
||||
|
||||
For good measure, a second video is geolocated following the same procedure:
|
||||
|
||||
|
||||
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Another view of the explosions in Beirut <a href="https://t.co/efT5VlpMkj">pic.twitter.com/efT5VlpMkj</a></p>— Borzou Daragahi 🖊🗒 (@borzou) <a href="https://twitter.com/borzou/status/1290678580897251330?ref_src=twsrc%5Etfw">August 4, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
|
||||
|
||||
|
||||
The second pier (highlighted in green) and the angle (in blue) serve as references:
|
||||
|
||||
|
||||
|
||||
The video was taken from the rooftop of a japanese restaurant called Clap Beirut (in red above). This is confirmed by a picture of the rooftop bar on google images, which matches the bar that can be seen at 0:02 in the twitter video. Below is a comparison of the video view and the 3D OSM model:
|
||||
|
||||
|
||||
|
||||
The video was taken from the rooftop of a Japanese restaurant called Clap Beirut (in red above). This is confirmed by a picture of the rooftop bar on google images, which matches the bar that can be seen at 0:02 in the twitter video. Below is a comparison of the video view and the 3D OSM model:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Though somewhat grainy, the basemap on the OSM rendering shows the same parking lot in the foreground, the second pier, and the same two buildings highlighted in yellow. Having created a 3D model of Beirut using OSM data, we can now simulate how the explosion would interact with the cityscape.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Using a Viewshed Analysis to Assess Blast Exposure
|
||||
|
||||
|
||||
As the pressure wave moved through the Beirut, some buildings bore the full force of the explosion, while others were partially shielded by taller structures. A viewshed analysis can be conducted to identify surfaces that were directly exposed to the explosion by creating a lighting object at ground zero; areas that are lit up experienced unobstructed exposure to the blast:
|
||||
|
||||
|
||||
|
||||
Pressure waves, like sound, are capable of diffraction (beding around small obstructions). To roughly simluate this, the lighting object is gradually raised, allowing the light to pass "around" obstructions. Warehouses on the Eastern side of the docks, as well as the first row of apartment buildings facing the docks are immediately affected. As the lighting object rises above the warehouse, more areas suffer direct exposure.
|
||||
|
||||
|
||||
|
||||
Pressure waves, like sound, are capable of diffraction (bending around small obstructions). To roughly simulate this, the lighting object is gradually raised, allowing the light to pass "around" obstructions. Warehouses on the Eastern side of the docks, as well as the first row of apartment buildings facing the docks are immediately affected. As the lighting object rises above the warehouse, more areas suffer direct exposure.
|
||||
|
||||
|
||||
Using two lighting objects-- a red one at 10 meters and a blue one at 20 meters above the warehouse at ground zero-- the intensity of the blast in different areas is highlighted; red areas suffered direct exposure, blue areas suffered partially obstructed exposure, and black areas were indirectly exposed.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
In the immediate vicinity of the explosion the large "L" shaped building (Lebanon's strategic grain reserve) is bright red, and was barely left standing. It absorbed a large amount of the blast, shielding areas behind it and thereby casting a long blue shadow to the West. If one refers back to the satellite damage maps above, there appears to be significantly less damage in the area just West of ("behind") the grain silo, roughly corresponding to the blue shadow above. While these areas were still heavily damaged, they seem to have suffered less damage than areas of equal distance to the East.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Accounting for Diffraction
|
||||
|
||||
|
||||
The viewshed analysis tells us which sides of a building are exposed to the blast, but it's a pretty rough approximation of the way the pressure wave would respond to obstacles in its path. As previously mentioned, pressure waves behave much like sound waves or waves in water: they bounce off of objects, move around obstructions, and gradually fade.
|
||||
|
||||
|
||||
To get a more precise idea of the way in which the blast interacted with the urban environment, we can model the blast as an actual wave using the "dynamic wave" feature in Blender. This effectively involves creating a two-dimensional plane, telling it to behave like water, and simulating an object being dropped into the water. By putting an obstruction in this plane, we can see how the wave responds to it. As an example, the grain silo has been isolated below:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
As the blast hits the side of the silo, it is reflected. Two large waves can be seen traveling to the right: the initial blast wave, and the reflection from the silo which rivals the initial wave in magnitude. To the left, the wave travels around the silo but is significantly weakened.
|
||||
|
||||
|
||||
Broadening the focus and adding the rest of the OSM data back in, we can observe how the pressure wave interacted with buildings on the waterfront:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The warehouses on the docks were omitted to emphasize the interaction between the pressure wave and the waterfront buildings; their light metal structure and low height means they would have caused little reflection anyway. The general pattern of the dynamic wave is consistent with the viewshed, but adds a layer of detail. The blast is reflected off of the silo towards the East, leading to a double hit. Though the wave still moves around the silo to the West, the pressure is diminished. Once the wave hits the highrises, the pattern becomes noisy as the wave both presses forward into the mainland and is reflected back towards the pier.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Modeling the Pressure Wave
|
||||
|
||||
Now that we've accounted for the directionality of the blast and the influence of buildings, we can model the pressure wave itself. An expanding sphere centered at ground zero is used to model the progression of the pressure wave through the city. To get a visual sense of the blast's force, the color of the sphere will be a function of the pressure exerted by pressure wave.
|
||||
|
||||
The pressure exerted by the explosion in kilopascals (kPa) at various distances can be calculated using the DoD's Blast Effects Computer, which allows users to input variables such as the TNT equivalent of the ordnance, storage method, and elevation. Though there are several estimates, the blast was likely equivalent to around 300 tons of TNT. The direct "incident pressure" of the pressure wave is shown in blue. However, pressure waves from explosions that occur on the ground are reflected upwards, amplifying the total pressure exerted by the blast. This "reflected pressure" is shown in orange:
|
||||
Now that we've accounted for the directionality of the blast and the influence of buildings, we can model the pressure wave itself. An expanding sphere centered at ground zero is used to model the progression of the pressure wave through the city. To get a visual sense of the blast's force, the color of the sphere will be a function of the pressure exerted by the pressure wave.
|
||||
|
||||
|
||||
The pressure exerted by the explosion in kilopascals (kPa) at various distances can be calculated using the U.S. Department of Defemse's Blast Effects Computer, which allows users to input variables such as the TNT equivalent of the ordnance, storage method, and elevation. Though there are several estimates, the blast was likely equivalent to around 300 tons of TNT. The direct "incident pressure" of the pressure wave is shown in blue. However, pressure waves from explosions that occur on the ground are reflected upwards, amplifying the total pressure exerted by the blast. This "reflected pressure" is shown in orange:
|
||||
|
||||
|
||||
<iframe title="Blast Overpressure and Distance " aria-label="Interactive line chart" id="datawrapper-chart-J1Pb1" src="https://datawrapper.dwcdn.net/J1Pb1/1/" scrolling="no" frameborder="0" style="width: 0; min-width: 100% !important; border: none;" height="400"></iframe><script type="text/javascript">!function(){"use strict";window.addEventListener("message",(function(a){if(void 0!==a.data["datawrapper-height"])for(var e in a.data["datawrapper-height"]){var t=document.getElementById("datawrapper-chart-"+e)||document.querySelector("iframe[src*='"+e+"']");t&&(t.style.height=a.data["datawrapper-height"][e]+"px")}}))}();
|
||||
</script>
|
||||
</script>
|
||||
|
||||
|
||||
For reference, 137 kPa results in 99% fatalities, 68 kPa is enough to cause structural damage to most buildings, and 20 kPa results in serious injuries. 1-6 kPa is enough to break an average window. At 1km, the reflected pressure of the blast (18 kPa) was still enough to seriously injure. Precisely calculating the force exerted by an explosion is exceptionally complicated, however, so these numbers should be treated as rough estimates. Further analysis of the damage caused by the blast can be derived from the UN's Explosion Consequences Analysis calculator which provides distance values for different types of damage and injuries.
|
||||
|
||||
For reference, 137 kPa results in 99% fatalities, 68 kPa is enough to cause structural damage to most buildings, and 20 kPa results in serious injuries. 1-6 kPa is enough to break an average window. At 1km, the reflected pressure of the blast (18 kPa) was still enough to seriously injure. Precisely calculating the force exerted by an explosion is exceptionally complicated, however, so these numbers should be treated as rough estimates. Further analysis of the damage caused by blasts blast can be derived from the UN's Explosion Consequences Analysis calculator which provides distance values for different types of damage and injuries.
|
||||
|
||||
Linking the values in this graph to the color of the pressure wave sphere provides a visual representation of the blast's force as it expands. An RGB color scale corresponds to the blast's overpressure at three threshold values.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
By keeping the lighting object from the viewshed analysis and placing it within the expanding sphere of the pressure wave, we combine two key pieces of information: the pressure exerted by the blast (the color of the sphere), and the level of directional exposure (brightness).
|
||||
|
||||
|
||||
Now, referring back to the two geolocated twitter videos from earlier, we can recreate the blast in our 3D model and get some new insights. Below is a side-by-side comparison of the first video and the 3D model:
|
||||
|
||||
|
||||
|
||||
Judging by the twitter video alone, it would be very hard to tell the fate of the person filming or the damage caused to the building that they were in. However, the 3D model shows that despite having an unobstructed view of the explosion, the incident pressure of the pressure wave had decreased significtantly by the time it reached the viewing point. The blue-green color corresponds to roughly 15 kPa-- enough to injure and break windows, but not enough to cause structural damage to the building.
|
||||
|
||||
|
||||
|
||||
Judging by the twitter video alone, it would be very hard to tell the fate of the person filming or the damage caused to the building that they were in. However, the 3D model shows that despite having an unobstructed view of the explosion, the incident pressure of the pressure wave had decreased significantly by the time it reached the viewing point. The blue-green color corresponds to roughly 15 kPa-- enough to injure and break windows, but not enough to cause structural damage to the building.
|
||||
|
||||
|
||||
The second twitter video was taken slightly closer to ground zero, but the view was partially obstructed by the grain silo:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Though the pressure wave probably exerted more pressure compared to the first angle, the partial obstruction of the grain silo likely tempered the force of the blast.
|
||||
|
||||
|
||||
### Assessing Damage to the Skyline Tower
|
||||
|
||||
|
||||
As a concrete example of how this approach can be used to assess damage (or predict it, if one had the foresight), let us consider the Skyline Tower, pictured below following the explosion:
|
||||
|
||||
|
||||
|
||||
This partial side view shows two faces of the building, labelled "A" and "B" above. Side A was nearly perpendicular to the blast, and just 600 m from ground zero. Based on the previous modeling, the pressure wave exerted roughly 40 kPa on this side of the building. The corner where sides A and B meet, highlighted in green, shows total destruction of windows, removal of most siding panels, and structural damage. The back corner, highlighted in red, shows many windows still intact, indicating that the maximum overpressure on this side of the building likely didn't exeed 10 kPa. In other words, standing on the front balcony would likely have led to serious injury but standing on the back balcony would have been relatively safe.
|
||||
|
||||
|
||||
|
||||
This partial side view shows two faces of the building, labeled "A" and "B" above. Side A was nearly perpendicular to the blast, and just 600m from ground zero. Based on the previous modeling, the pressure wave exerted roughly 40 kPa on this side of the building. The corner where sides A and B meet, highlighted in green, shows total destruction of windows, removal of most siding panels, and structural damage. The back corner, highlighted in red, shows many windows still intact, indicating that the maximum overpressure on this side of the building likely didn't exceed 10 kPa. In other words, standing on the front balcony would likely have led to serious injury but standing on the back balcony would have been relatively safe.
|
||||
|
||||
|
||||
The animation below shows the Skyline Tower as it is hit by the pressure wave, with sides A and B labeled:
|
||||
|
||||
|
||||
|
||||
The bright green color of the pressure wave indicates a strong likelihood of structural damage. Side A can be seen taking a direct hit, while side B is angled slighly away. Despite not being directly exposed to the blast, it likely still took reflective damage from some of the neighbouring buildings. Both the incident overpressure indicated by the color of the sphere, as well as the relative brightness of sides A and B both correspond closely to the observed damage taken by the Skyline Tower.
|
||||
|
||||
|
||||
|
||||
The bright green color of the pressure wave indicates a strong likelihood of structural damage. Side A can be seen taking a direct hit, while side B is angled slightly away. Despite not being directly exposed to the blast, it likely still took reflective damage from some of the neighboring buildings. Both the incident overpressure indicated by the color of the sphere, as well as the relative brightness of sides A and B both correspond closely to the observed damage taken by the Skyline Tower.
|
||||
|
||||
|
||||
### Further Research
|
||||
|
||||
|
||||
Though satellite imagery analysis is an indispensable tool in disaster response, it has limitations. Urban blast damage in particular is difficult to assess accurately because it is highly directional and much of it cannot be seen from a bird's eye view. Using free and open source tools, an interactive 3D model of an urban explosion can be generated, allowing for a highly detailed investigation of directional blast damage. This can be achieved in three steps:
|
||||
|
||||
|
||||
First, creating a 3D model of the urban area using Blender and Open Street Maps data. Second, conducting a viewshed analysis using lighting objects to gauge levels of unobstructed exposure to the pressure wave. Third, modeling the explosion using geolocated videos of the event and ordnance calculators. For added detail, a dynamic wave analysis can be used to more precisely model how the pressure wave interacts with buildings.
|
||||
|
||||
Once properly modeled, the explosion can be viewed from any angle in the city. The viewshed analysis can be calibrated more finely by ground-truthing various damage levels (e.g. broken windows) at different locations. In the absence of an official address registry in Beirut, OSM is already being used by the Lebanese Red Cross (donate here) to conduct neighborhood surveys assessing blast damage. As such, this type of damage analysis can quickly be integrated into relief efforts, adapted to model disasters in different cities, and can even be used to simulate the destructive potential of hypothetical explosions to promote readiness.
|
||||
|
||||
Speacial thanks to my nuclear physicist brother, Sean, for making sure I didn't commit too many crimes against Physics.
|
||||
Once properly modeled, the explosion can be viewed from any angle in the city. The viewshed analysis can be calibrated more finely by ground-truthing various damage levels (e.g. broken windows) at different locations. In the absence of an official address registry in Beirut, OSM is already being used by the Lebanese Red Cross (donate here) to conduct neighborhood surveys assessing blast damage. As such, this type of damage analysis can quickly be integrated into relief efforts, adapted to model disasters in different cities, and can even be used to simulate the destructive potential of hypothetical explosions to promote readiness.
|
||||
@@ -1,49 +1,71 @@
|
||||
# Ship Detection {.unnumbered}
|
||||
---
|
||||
title: Ship Detection
|
||||
---
|
||||
|
||||
|
||||
There's a huge amount of data available on the internet about ship movements, most of which draws on the Automatic Identification System (AIS) which uses radio to broadcast the identity, position, course, speed and other data about ships. [MarineTraffic](https://www.marinetraffic.com/en/ais-api-services), for example, provides an API that allows you to query the location of ships in real time as well as historical vessel tracks and lots of other useful data. Unfortunately most sources of AIS data are paywalled, and AIS can be turned off or manipulated to hide the identity or position of the ship. In fact, most of the stuff we're interested in investigating probably happens when AIS is turned off.
|
||||
|
||||
There's a huge amount of data available on the internet about ship movements, most of which draw on the Automatic Identification System (AIS) which is a system that uses radio to broadcast the identity, position, course, speed, and other data about ships. [MarineTraffic](https://www.marinetraffic.com/en/ais-api-services), for example, provides an API that allows you to query the location of ships in real time as well as historical vessel tracks and lots of other useful data. Unfortunately most sources of AIS data are paywalled, and AIS can be turned off or manipulated to hide the identity or position of the ship. In fact, most of the stuff we're interested in investigating probably happens when AIS is turned off.
|
||||
|
||||
Though ships can hide by turning off their AIS transponders, they can't hide from satellites. In this tutorial, we're going to build an application that uses Synthetic Aperture Radar (SAR) from the European Space Agency's Sentinel-1 satellite to automatically identify ships, regardless of whether they've got their transponders turned on or off. Here's the finished application:
|
||||
|
||||
|
||||
:::{.column-page}
|
||||
|
||||
|
||||
<iframe src='https://ollielballinger.users.earthengine.app/view/shipdetection' width='100%' height='700px'></iframe>
|
||||
|
||||
|
||||
:::
|
||||
|
||||
|
||||
## How it Works
|
||||
|
||||
|
||||
The app has two main panels:
|
||||
|
||||
|
||||
1. A control panel on the left that allows the user to interact with the application
|
||||
2. A map on the right that displays the results
|
||||
|
||||
|
||||
The control panel has a date slider that allows the user to load imagery from a particular year. Below that is a graph that shows the number of ships detected over time within that year. A slider underneath the graph lets us toggle the sensitivity of the ship detection process. Finally, a button at the bottom lets the user draw their own area of interest on the map, and the app will automatically detect ships within that area.
|
||||
|
||||
The map panel visualizes the results of the ship detection process and has three layers. The bottom layer is the Sentinel-1 image that we're using to detect ships; it's blue/purple, and if you zoom in and look closley you can see bright specks in the sea, which are ships. When Sentinel-1 sends a pulse of radio waves onto a flat surface like the sea, there is very little to reflect the waves back to the satellite-- they just bounce off into space. A low return signal means we'll see a darker color on our map. But when the raio waves hit a ship they are reflected back to the satelite and generate a higher return signal, and therefore a much brighter color. The second layer on the map displays a bunch of green points; each one of these is a detected ship. The last layer shows the red outline of the area of interest that the user drew on the map. You can zoom in on the map by holding down the command button and scrolling up and down.
|
||||
|
||||
The map panel visualizes the results of the ship detection process and has three layers. The bottom layer is the Sentinel-1 image that we're using to detect ships; it's blue/purple, and if you zoom in and look cloesly you can see bright specks in the sea, which are ships. When Sentinel-1 sends a pulse of radio waves onto a flat surface like the sea, there is very little to reflect the waves back to the satellite -- they just bounce off into space. A low return signal means we'll see a darker color on our map. But when the radio waves hit a ship they are reflected back to the satellite and generate a higher return signal, and therefore a much brighter color. The second layer on the map displays a bunch of green points; each one of these is a detected ship. The last layer shows the red outline of the area of interest that the user drew on the map. You can zoom in on the map by holding down the command button and scrolling up and down.
|
||||
|
||||
|
||||
When the application is first loaded it is centered on an area just north of the Suez Canal, and is analyzing imagery from 2021. We can see a bunch of green dots in the AOI, which is the main waiting area for ships waiting to transit the canal. It's a bit crowded because it's visualizing all of the ships detected in the entire year. We can display imagery from a single day by clicking on a point in the graph on the left, which you will notice displays a huge spike in the number of ships detected around March.
|
||||
|
||||
You might remember that on March 23rd, 2021, the Ever Given-- a 400m long container ship-- got stuck in the Suez Canal. The ship was blocking the canal for six days, and it's estimated that it cost the global economy $400 million per day. If you click on the tip of the spike on March 30th, you can see backup of around 150 ships waiting for the canal to be cleared. You can also zoom in on a particular date range by scrolling and dragging on the graph. If you zoom in on the spike, you can then select imagery from early April to compare the number of ships in the waiting area after the blockage was cleared. In normal times we can see a regular pattern in the number of ships in the waiting area ranging between 15 and 40 ships.
|
||||
|
||||
You might remember that on March 23rd, 2021, the Ever Given -- a 400m long container ship -- got stuck in the Suez Canal. The ship was blocking the canal for six days, and it's estimated that it cost the global economy $400 million per day. If you click on the tip of the spike on March 30th, you can see a backup of around 150 ships waiting for the canal to be cleared. You can also zoom in on a particular date range by scrolling and dragging on the graph. If you zoom in on the spike, you can then select imagery from early April to compare the number of ships in the waiting area after the blockage was cleared. In normal times we can see a regular pattern in the number of ships in the waiting area ranging between 15 and 40 ships.
|
||||
|
||||
|
||||
If you're closely zoomed in to the map and load imagery from different days by clicking on the graph, you can compare the bright spots on the Sentinel image and the green dots. The ship detection process is pretty accurate, and we typically see one green dot per ship. However, you may notice that we occasionally miss a ship. This is because the ship detection process is based on a threshold, and if the ship is too small it may not generate a high enough return signal to be detected. You can increase the sensitivity of the ship detection process by moving the slider below the graph. This will increase the number of ships detected, but it may also increase the number of false positives.
|
||||
|
||||
|
||||
The next section focuses on building this application. After that, we'll have a look at a few different use cases for this sort of maritime surveillance.
|
||||
|
||||
|
||||
# Building the Application
|
||||
|
||||
|
||||
## Setup
|
||||
|
||||
|
||||
The first step is to configure the map and import the necessary datasets. By default, we want the app to be centered on the Suez Canal. Then, we want to import the Digital Surface Model (DSM) from the ALOS World 3D-30 dataset. This dataset provides a 30m resolution elevation model of the Earth which we will use to mask out the land. Finally, we want to import the Sentinel 1 dataset. We will use the VV polarization and the Interferometric Wide (IW) mode. We will also sort the images by date.
|
||||
|
||||
|
||||
```js
|
||||
// Center the map on the Suez Canal and set map options
|
||||
Map.setCenter(32.327, 31.4532, 10);
|
||||
Map.setOptions("Hybrid");
|
||||
Map.setControlVisibility({ all: false });
|
||||
|
||||
|
||||
// Import the Digital Surface Model (DSM) from the ALOS World 3D-30 dataset
|
||||
var dem = ee.ImageCollection("JAXA/ALOS/AW3D30/V3_2").mean().select("DSM");
|
||||
|
||||
|
||||
// Import the Sentinel 1 dataset
|
||||
var s1 = ee
|
||||
.ImageCollection("COPERNICUS/S1_GRD")
|
||||
@@ -51,6 +73,7 @@ var s1 = ee
|
||||
.filter(ee.Filter.eq("instrumentMode", "IW"))
|
||||
.sort("system:time_start");
|
||||
|
||||
|
||||
// Define the default area of interest
|
||||
var suez = ee.Geometry.Polygon([
|
||||
[
|
||||
@@ -62,24 +85,31 @@ var suez = ee.Geometry.Polygon([
|
||||
]);
|
||||
```
|
||||
|
||||
|
||||
Now that we've gotten that out of the way, we can move on to the actual detection of ships.
|
||||
|
||||
|
||||
## Ship Detection
|
||||
|
||||
|
||||
You might expect the automatic identification of ships based on synthetic aperture radar satellite imagery to involve a complex machine learning algorithm or artificial intelligence. In fact, it can be done in one line of code which sets a cutoff. If the return signal is greater than 0, then we have a ship. If it's less than 0, then we don't. Simple as that.
|
||||
|
||||
|
||||
The main analytical function responsible for ship identification is the `getVectors` function shown below. It takes an image as an input and returns a FeatureCollection of points, each corresponding to a ship. The function clips the image to the area of interest, selects the VV polarization, and finally filters out areas where the VV value is smaller than 0. This results in a raster image where the sea is black and the ships are white. We then use the `reduceToVectors` function to convert the raster image to a FeatureCollection of points. The function returns this FeatureCollection, and sets a property called `count` which is the number of ships detected in the image.
|
||||
|
||||
|
||||
```js
|
||||
function getVectors(img) {
|
||||
// Get the area of interest from the drawing tools widget.
|
||||
var aoi = drawingTools.layers().get(0).getEeObject();
|
||||
|
||||
|
||||
// Clip the image to the area of interest
|
||||
// Select the VV polarization
|
||||
// Filter areas where the VV value is greater than 0
|
||||
var cutoff = img.clip(aoi).select("VV").gt(0)
|
||||
|
||||
|
||||
// Convert the raster image to a FeatureCollection of points
|
||||
var points = cutoff.reduceToVectors({
|
||||
geometry: aoi,
|
||||
@@ -89,6 +119,7 @@ function getVectors(img) {
|
||||
maxPixels: 1653602926,
|
||||
});
|
||||
|
||||
|
||||
// Set the number of ships detected in the image as a property called "count"
|
||||
var count = points.size();
|
||||
// Set the date of the image as a property called "system:time_start"
|
||||
@@ -98,15 +129,19 @@ function getVectors(img) {
|
||||
```
|
||||
The `count` and `system:time_start` properties are used to create the graph of daily ship counts and allow the resulting vector (point) data to interact with the date slider widget. An important detail here is that the "scale" parameter of the `reduceToVectors` function is set to the value of the scale slider widget. This allows the user to adjust the resolution of the ship detection process; a smaller value will allow us to detect smaller ships.
|
||||
|
||||
|
||||
## Visualization
|
||||
|
||||
The `viz` function is responsible for displaying the results of the ship detection process. It takes the area of interest, the vector data, and the Sentinel 1 image as inputs. Nothing super complicated here; we're just creating three layers and adding them to the map in order: the underlying Sentinel-1 image raster, the ship vector data in green, and the area of interest outline in red. We're using the `Map.layers().set()` function to replace the existing layers with the new ones, rather than addine new ones each time.
|
||||
|
||||
The `viz` function is responsible for displaying the results of the ship detection process. It takes the area of interest, the vector data, and the Sentinel 1 image as inputs. Nothing super complicated here; we're just creating three layers and adding them to the map in order: the underlying Sentinel-1 image raster, the ship vector data in green, and the area of interest outline in red. We're using the `Map.layers().set()` function to replace the existing layers with the new ones, rather than adding new ones each time.
|
||||
|
||||
|
||||
```js
|
||||
function viz(aoi, vectors, s1Filtered) {
|
||||
// Create an empty image into which to paint the features, cast to byte.
|
||||
var empty = ee.Image().byte();
|
||||
|
||||
|
||||
// Paint all the polygon edges with the same number and width, display.
|
||||
var outline = empty.paint({
|
||||
featureCollection: aoi,
|
||||
@@ -114,9 +149,11 @@ function viz(aoi, vectors, s1Filtered) {
|
||||
width: 3,
|
||||
});
|
||||
|
||||
|
||||
// Create a layer for the area of interest in red
|
||||
var aoi_layer = ui.Map.Layer(outline, { palette: "red" }, "AOI");
|
||||
|
||||
|
||||
// Create a layer for the vector data in green
|
||||
var vectorLayer = ui.Map.Layer(
|
||||
vectors.flatten(),
|
||||
@@ -124,6 +161,7 @@ function viz(aoi, vectors, s1Filtered) {
|
||||
"Vectors"
|
||||
);
|
||||
|
||||
|
||||
// Create a layer for the Sentinel 1 image in false color
|
||||
var sarLayer = ui.Map.Layer(
|
||||
s1Filtered,
|
||||
@@ -131,17 +169,21 @@ function viz(aoi, vectors, s1Filtered) {
|
||||
"SAR"
|
||||
);
|
||||
|
||||
|
||||
// Add the layers in order
|
||||
Map.layers().set(0, sarLayer);
|
||||
Map.layers().set(1, vectorLayer);
|
||||
Map.layers().set(2, aoi_layer);
|
||||
}
|
||||
```
|
||||
We want a function to handle the visualization because there are two different situations in which we're going to visualize results, and we dont want to repeat our code. The first situation is when the user draws a new area of interest, moves the date slider, or alters the scale. In this case, we want to visualize the results of the ship detection process for the entire year's worth of Sentinel-1 imagery. The second situation is when the user clicks on the chart to analyze a particular day. In this case, we obviously only want to visualize the results of the ship detection process on that day. With this function, we can simply pass the appropriately filtered versions of the Sentinel-1 image and vector data to the function, and it will visualize the results, rather than having to write the same code twice.
|
||||
We want a function to handle the visualization because there are two different situations in which we're going to visualize results, and we don’t want to repeat our code. The first situation is when the user draws a new area of interest, moves the date slider or alters the scale. In this case, we want to visualize the results of the ship detection process for the entire year's worth of Sentinel-1 imagery. The second situation is when the user clicks on the chart to analyze a particular day. In this case, we obviously only want to visualize the results of the ship detection process on that day. With this function, we can simply pass the appropriately filtered versions of the Sentinel-1 image and vector data to the function, and it will visualize the results, rather than having to write the same code twice.
|
||||
|
||||
|
||||
## Putting it all together
|
||||
|
||||
Having defined a few helper functions to handle the visualization and ship detection process, we can now move on to the main function that will perform the analysis. This will be performed by the `daterangeVectors` function. In a nutshell, it read the user specified date range from the date slider widget, and filter the Sentinel 1 dataset to only include images within that period. Then, it will loop through each Sentinel-1 image from that year and apply the `getVectors` function to count the number of ships that fall within the area of interest and generate a dataset of points corresponding to detected ships. We'll then use the `viz` function we just defined to visualize the all of the ship detections and Sentinel-1 images in the AOI during that year stacked on top of each other. Finally, we'll create a chart based on the number of ships detected per day, and allow the user to click on the chart to visualize the results for a particular day.
|
||||
|
||||
Having defined a few helper functions to handle the visualization and ship detection process, we can now move on to the main function that will perform the analysis. This will be performed by the `daterangeVectors` function. In a nutshell, it reads the user specified date range from the date slider widget, and filters the Sentinel 1 dataset to only include images within that period. Then, it will loop through each Sentinel-1 image from that year and apply the `getVectors` function to count the number of ships that fall within the area of interest and generate a dataset of points corresponding to detected ships. We'll then use the `viz` function we just defined to visualize all of the ship detections and Sentinel-1 images in the AOI during that year stacked on top of each other. Finally, we'll create a chart based on the number of ships detected per day, and allow the user to click on the chart to visualize the results for a particular day.
|
||||
|
||||
|
||||
```js
|
||||
var daterangeVectors = function () {
|
||||
@@ -152,21 +194,26 @@ var daterangeVectors = function () {
|
||||
ee.Date(dateSlider.getValue()[1])
|
||||
);
|
||||
|
||||
|
||||
// Get the area of interest from the drawing tools widget.
|
||||
var aoi = drawingTools.layers().get(0).getEeObject();
|
||||
|
||||
|
||||
// Hide the user-drawn shape.
|
||||
drawingTools.layers().get(0).setShown(false);
|
||||
|
||||
|
||||
// Filter the Sentinel 1 dataset to only include images within the date range, and within the area of interest.
|
||||
var s1Filtered = s1.filterDate(range.start(), range.end()).filterBounds(aoi);
|
||||
|
||||
// Count the number of ships in each image using the getVectors function
|
||||
var vectors = s1Filtered.map(getVectors);
|
||||
|
||||
|
||||
// Use the viz function to visualize the results
|
||||
viz(aoi, vectors, s1Filtered.max().updateMask(dem.lte(0)));
|
||||
|
||||
|
||||
// Create a chart of the number of ships per day
|
||||
var chart = ui.Chart.feature
|
||||
.byFeature({
|
||||
@@ -182,55 +229,71 @@ var daterangeVectors = function () {
|
||||
series: "Area of Interest",
|
||||
});
|
||||
|
||||
|
||||
// Add the chart at a fixed position, so that new charts overwrite older ones.
|
||||
controlPanel.widgets().set(4, chart);
|
||||
|
||||
|
||||
// Add a click handler to the chart to filter the map by day.
|
||||
chart.onClick(filterDay);
|
||||
};
|
||||
```
|
||||
|
||||
There's one function referenced above-- `filterDay`-- that we haven't defined yet. This function is called when the user clicks on the chart to analyze a particular day. It takes the date of the clicked day as an input, filters the Sentinel-1 dataset and vector data accordingly, and uses the `viz` function to display the results for that day.
|
||||
|
||||
There's one function referenced above -- `filterDay`-- that we haven't defined yet. This function is called when the user clicks on the chart to analyze a particular day. It takes the date of the clicked day as an input, filters the Sentinel-1 dataset and vector data accordingly, and uses the `viz` function to display the results for that day.
|
||||
|
||||
|
||||
```js
|
||||
function filterDay (callback) {
|
||||
|
||||
|
||||
// Get the date of the clicked day
|
||||
var date = ee.Date(callback);
|
||||
|
||||
|
||||
// Filter the vector data to only include images from that day
|
||||
var vectorDay = vectors.filterDate(date);
|
||||
|
||||
// Filter the Sentinel-1 imagery to only include images from that day
|
||||
var s1Day = s1.filterDate(date).max().updateMask(dem.lte(0));
|
||||
|
||||
|
||||
// Use the viz function to visualize the results
|
||||
viz(aoi, vectorDay, s1Day);
|
||||
};
|
||||
```
|
||||
|
||||
|
||||
The analytical portion of the application is now complete. Now we have to build a user interface that lets us interact with the application.
|
||||
|
||||
|
||||
## Building a User Interface
|
||||
|
||||
|
||||
There are four main steps in the process of creating the User Interface (UI):
|
||||
|
||||
|
||||
1. Configure the drawing tools that allow the user to draw a polygon on the map.
|
||||
2. Create some widgets
|
||||
|
||||
|
||||
### Drawing Tools
|
||||
|
||||
|
||||
We eventually want to allow the user to draw a polygon on the map, and count the number of ships that fall within it. In order to do so, we need to set up a few functions related to the drawing tools that allow the user to do this. Among other things, we want to make sure that we're clearing the old geometries so that we're only ever conducting analysis inside the most recent user-drawn polygon, so we'll need to clear the old ones. We also want to specify the type of polygon the user can draw, which for ease will be a rectangle (you could change this to the actual "polygon" type if you wanted to draw more complex geometries).
|
||||
|
||||
|
||||
```js
|
||||
var drawingTools = Map.drawingTools();
|
||||
|
||||
|
||||
// Remove any existing layers
|
||||
while (drawingTools.layers().length() > 0) {
|
||||
var layer = drawingTools.layers().get(0);
|
||||
drawingTools.layers().remove(layer);
|
||||
}
|
||||
|
||||
|
||||
// Add a dummy layer to the drawing tools object (the Suez Canal box)
|
||||
var dummyGeometry = ui.Map.GeometryLayer({
|
||||
geometries: null,
|
||||
@@ -238,10 +301,13 @@ var dummyGeometry = ui.Map.GeometryLayer({
|
||||
.fromGeometry(suez)
|
||||
.setShown(false);
|
||||
|
||||
|
||||
// Add the dummy layer to the drawing tools object
|
||||
drawingTools.layers().add(dummyGeometry);
|
||||
|
||||
|
||||
|
||||
|
||||
// Create a function that clears existing geometries and lets the user draw a rectangle
|
||||
function drawPolygon() {
|
||||
var layers = drawingTools.layers();
|
||||
@@ -251,9 +317,12 @@ function drawPolygon() {
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
### Widgets
|
||||
|
||||
The control panel will eventually contain a few different widgets that allow the user to interact with the application. We'll start by creating a button that allows the user to draw a polygon on the map. We'll also create a slider that allows the user to adjust the size of the ships that are detected (remember, this manipualtes the "scale" parameter in the `reduceToVectors` function used in the detection process). The slider will have an accompanying label that tells the user what it does.
|
||||
|
||||
The control panel will eventually contain a few different widgets that allow the user to interact with the application. We'll start by creating a button that allows the user to draw a polygon on the map. We'll also create a slider that allows the user to adjust the size of the ships that are detected (remember, this manipulates the "scale" parameter in the `reduceToVectors` function used in the detection process). The slider will have an accompanying label that tells the user what it does.
|
||||
|
||||
|
||||
```js
|
||||
// Create a button that allows the user to draw a polygon on the map
|
||||
@@ -263,6 +332,7 @@ var drawButton = ui.Button({
|
||||
style: { stretch: "horizontal" },
|
||||
});
|
||||
|
||||
|
||||
// Create a slider that allows the user to adjust the size of the ships that are detected
|
||||
var scaleSlider = ui.Slider({
|
||||
min: 1,
|
||||
@@ -273,9 +343,11 @@ var scaleSlider = ui.Slider({
|
||||
style: { width: "70%" },
|
||||
});
|
||||
|
||||
|
||||
// Create a label for the slider
|
||||
var scaleLabel = ui.Label("Ship Size: ");
|
||||
|
||||
|
||||
// Create a panel that contains the slider and its label
|
||||
var scalePanel = ui.Panel({
|
||||
widgets: [scaleLabel, scaleSlider],
|
||||
@@ -284,13 +356,16 @@ var scalePanel = ui.Panel({
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
The last widget we're going to define is the date slider. This widget will trigger the `daterangeVectors` function, which will filter the Sentinel-1 dataset to only include images from the selected year, and then run the detection process on the filtered dataset.
|
||||
|
||||
|
||||
```js
|
||||
// Specify the start and end dates for the date slider
|
||||
var start = "2014-01-01";
|
||||
var now = Date.now();
|
||||
|
||||
|
||||
// Create a date slider that allows the user to select a year
|
||||
var dateSlider = ui.DateSlider({
|
||||
value: "2021-03-01",
|
||||
@@ -302,9 +377,12 @@ var dateSlider = ui.DateSlider({
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
### The Control Panel
|
||||
|
||||
Now we're going to assemble all of the widgets we've just defined into one panel, alongsie some explanatory text. I'm adding a blank label to the panel as a placeholder for the chart, since it will be re-added to the panel every time the user changed the date on the date slider, the AOI, or the scale.
|
||||
|
||||
Now we're going to assemble all of the widgets we've just defined into one panel, alongside some explanatory text. I'm adding a blank label to the panel as a placeholder for the chart, since it will be re-added to the panel every time the user changes the date on the date slider, the AOI, or the scale.
|
||||
|
||||
|
||||
```js
|
||||
var controlPanel = ui.Panel({
|
||||
@@ -330,42 +408,58 @@ var controlPanel = ui.Panel({
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
Once the control panel has been defined, we can add it to
|
||||
|
||||
|
||||
```js
|
||||
// Add the control panel to the map
|
||||
ui.root.insert(0,controlPanel);
|
||||
|
||||
|
||||
// Trigger the daterangeVectors function when the user draws a polygon
|
||||
drawingTools.onDraw(ui.util.debounce(daterangeVectors, 500));
|
||||
|
||||
|
||||
// Run the daterangeVectors function to initialize the map
|
||||
daterangeVectors();
|
||||
```
|
||||
|
||||
|
||||
And there we have it. A fully functional, all weather, daytime/nighttime ship detection tool that doesn't rely on AIS data. Let's play around with it.
|
||||
|
||||
|
||||
## Taking it for a spin
|
||||
|
||||
|
||||
### North Korea
|
||||
|
||||
In 2020, North Korea implemented one of the most severe COVID-19 lockdowns in the world including a near-total ban on ["all cross-border exchanges, including trade, traffic, and tourism".](https://thediplomat.com/2023/01/north-korea-likely-to-lift-pandemic-border-restrictions-in-2023/). Measures have been so severe that country appears to have experienced a significant [famine](https://foreignpolicy.com/2022/05/16/kim-north-korea-covid-outbreak-pandemic/). Though there were signs that things have gradually returned to normal, information on North Korea's economy is pretty hard to come by. Ship traffic in and out of the country's largest port, Nampo, is probably a pretty good indicator of the country's economic activity.
|
||||
|
||||
But we can't just head on down to Marine Tracker or other services that use AIS data to track ship movements. According to the [U.S. Treasury](https://home.treasury.gov/system/files/126/dprk_vessel_advisory_02232018.pdf), "North Korean-flagged merchant vessels have been known to intentionally disable their AIS transponders to mask their movements. This tactic, whether employed by North Korean-flagged vessels or other vessels involved in trade with North Korea, could conceal the origin or destination of cargo destined for, or originating in, North Korea." They should know-- they're the ones imposing the sanctions that make it illegal to trade with North Korea.
|
||||
In 2020, North Korea implemented one of the most severe COVID-19 lockdowns in the world including a near-total ban on ["all cross-border exchanges, including trade, traffic, and tourism".](https://thediplomat.com/2023/01/north-korea-likely-to-lift-pandemic-border-restrictions-in-2023/). Measures have been so severe that the country appears to have experienced a significant [famine](https://foreignpolicy.com/2022/05/16/kim-north-korea-covid-outbreak-pandemic/). Though there were signs that things have gradually returned to normal, information on North Korea's economy is pretty hard to come by. Ship traffic in and out of the country's largest port, Nampo, is probably a pretty good indicator of the country's economic activity.
|
||||
|
||||
|
||||
But we can't just head on down to Marine Tracker or other services that use AIS data to track ship movements. According to the [U.S. Treasury](https://home.treasury.gov/system/files/126/dprk_vessel_advisory_02232018.pdf), "North Korean-flagged merchant vessels have been known to intentionally disable their AIS transponders to mask their movements. This tactic, whether employed by North Korean-flagged vessels or other vessels involved in trade with North Korea, could conceal the origin or destination of cargo destined for, or originating in, North Korea." They should know -- they're the ones imposing the sanctions that make it illegal to trade with North Korea.
|
||||
|
||||
|
||||
A New York Times [investigation](https://www.nytimes.com/2019/07/16/world/asia/north-korea-luxury-goods-sanctions.html) tracked the maritime voyage of luxury Mercedes cars from Germany to North Korea via the Netherlands, China, Japan, South Korea, and Russia. AIS transponders were turned off at several points throughout this journey, and the investigation had to rely on satellite imagery to fill in the gaps.
|
||||
|
||||
|
||||
Though they used high resolution optical imagery to follow individual ships, we want to identify lots of ships in a large area over a long period. That would get very expensive, and automatic ship detection in optical imagery is relatively difficult. Here's how our SAR tool fares when we draw a box in the bay of Nampo:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Looking at imagery from 2021, we can see ship traffic increasing from nearly zero to around 40 ships per day.
|
||||
|
||||
|
||||
### Ukraine
|
||||
|
||||
|
||||
Odessa is Ukraine's largest port. Following its invasion of Ukraine in February 2022, Russia instituted a naval blockade against Ukrainian ports. The impact of this blockade is clearly visible using the tool we've just built:
|
||||
|
||||
|
||||
|
||||
|
||||
The daily number of ships detected in the port of Odessa dropped from 40-50 to 0-5 following the invasion, and remained near zero until the blockade was lifted in September 2022.
|
||||
|
||||
The daily number of ships detected in the port of Odessa dropped from 40 to 50 to 0 to 5 following the invasion, and remained near zero until the blockade was lifted in September 2022.
|
||||
@@ -1,165 +1,238 @@
|
||||
# Object Detection {.unnumbered}
|
||||
---
|
||||
title: Object Detection
|
||||
---
|
||||
|
||||
The Ship Detection tutorial explored a use case in which we might want to monitor the activity of ships in a particular location. That was a fairly straightforward task: the sea is very flat, and ships (especially large cargo and military vessels) protrude significantly. Using radar imagery, we could just set a threshold because if anything on the water is reflecting radio waves, it's probably a ship.
|
||||
|
||||
|
||||
One shortcoming of this approach is that it doesn't tell us what *kind* of ship we've detected. Sure, you could use the shape and size to distinguish between a fishing vessel and an aircraft carrier. But what about ships of similar sizes? Or what if you wanted to use satellite imagery to identify things other than ships, like airplanes, cars, or bridges? This sort of task-- called **"object detection"** is a bit more complicated.
|
||||
|
||||
|
||||
In this tutorial, we'll be using a deep learning model called **YOLOv5** to detect objects in satellite imagery. We'll be training the model on a custom dataset, and then using it to dynamically identify objects in satellite imagery of different resolutions pulled from Google Earth Engine. The tutorial is broken up into three sections:
|
||||
|
||||
|
||||
1. Object detection in satellite imagery
|
||||
2. Training a deep learning model on a custom dataset
|
||||
3. Dynamic inference using Google Earth Engine
|
||||
|
||||
|
||||
Unlike previous tutorials which used the GEE JavaScript API, **this one will utilize Python**; this is because these sorts of deep learning models aren't available in GEE natively yet. By the end, we'll be able to generate images such as the one below:
|
||||
|
||||
|
||||
:::{.column-screen}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::
|
||||
|
||||
|
||||
## Object Detection in Satellite Imagery
|
||||
|
||||
Object detction in satellite imagery has a variety of useful applications.
|
||||
|
||||
Object detection in satellite imagery has a variety of useful applications.
|
||||
|
||||
|
||||
There's the needle-in-a-haystack problem of needing to monitor a large area for a small number of objects. Immediately prior to the invasion of Ukraine, for example, a number of articles emerged showing Russian military vehicles and equipment popping up in small clearings in the forest near the border with Ukraine. Many of these deployments were spotted by painstakingly combing through high resolution satellite imagery, looking for things that look like trucks. One problem with this approach is that you need to know roughly where to look. The second, and more serious problem, is that you need to be on the lookout in the first place. Object detection, applied to satellite imagery, can automatically comb through vast areas and identify objects of interest. If planes and trucks start showing up in unexpected places, you'll know about it.
|
||||
|
||||
|
||||
Perhaps you're not monitoring that large of an area, but you want frequent updates about what's going on. What sorts of objects (planes, trucks, cars, etc.) are present? How many of each? Where are they located? Instead of having to manually look through new imagery as it becomes available, you could have a model automatically analyze new collections and output a summary.
|
||||
|
||||
|
||||
### YOLOv5
|
||||
|
||||
|
||||
Object detection is a fairly complicated task, and there are a number of different approaches to it. In this tutorial, we'll be using a model called **YOLOv5**. YOLO stands for **You Only Look Once**, and it's a model that was developed by [Joseph Redmon](https://pjreddie.com/) et. al., and the full paper detailing the model can be found [here](https://arxiv.org/abs/1506.02640).
|
||||
|
||||
|
||||
The YOLOv5 model is a **convolutional neural network** (CNN), which is a type of deep learning model. CNNs are very good at identifying patterns in images, particularly in small regions of images. This is important for object detection, because we want to be able to identify objects even if they're partially obscured by other objects.
|
||||
|
||||
|
||||
YOLO works by chopping an image up into a grid, and then predicting the location and size of objects in each grid cell:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
It learns the locations of these objects by training on a dataset of images in which each object is indicated by a **bounding box**. Then, when it's shown a new image, it will attempt to predict bounding boxes around the objects in that image. The standard YOLO model is trained on the [COCO dataset](https://cocodataset.org/#home), which contains over 200,000 images of 80 different objects ranging from people to cars to dogs. YOLO models pre-trained on this dataset work great out of the box to detect objects in videos, photographs, and live streams. But the nature of the objects we're interested in is a bit different.
|
||||
|
||||
|
||||
Luckily, we can simply re-train the YOLOv5 model on datasets of labeled satellite imagery. The rest of this tutorial will walk through the process of training YOLOv5 on a custom dataset, and then using it to dynamically identify objects in satellite imagery pulled from Google Earth Engine.
|
||||
|
||||
|
||||
## Training
|
||||
|
||||
|
||||
The process of re-training the YOLOv5 model on satellite imagery is fairly straightforward and can be accomplished in just three steps; first, we're going to clone the YOLOv5 repository which contains the model code and the training scripts. Then, we'll download a dataset of satellite imagery and labels from Roboflow, and finally, we'll train the model on that dataset.
|
||||
|
||||
|
||||
Let's start by cloning the YOLOv5 repository. Note: we'll be using a fork of the original repository that I've modified to include some pre-trained models that we'll be using later on.
|
||||
|
||||
|
||||
```python
|
||||
!git clone https://github.com/oballinger/yolov5_RS # clone repo
|
||||
%cd yolov5_RS # change directory to repo
|
||||
%pip install -qr requirements.txt # install dependencies
|
||||
%pip install -q roboflow # install roboflow
|
||||
|
||||
|
||||
import torch # install pytorch
|
||||
import os # for os related operations
|
||||
from IPython.display import Image, clear_output # to display images
|
||||
```
|
||||
|
||||
Once we've downloaded the YOLOv5 repository, we'll need to download a dataset of labelled satellite imagery. For this example, we're going to stick with ship detection as our use case, but expand upon it. We want to be able to distinguish between different types of ships, and we want to use freely-available satellite imagery.
|
||||
|
||||
Once we've downloaded the YOLOv5 repository, we'll need to download a dataset of labeled satellite imagery. For this example, we're going to stick with ship detection as our use case, but expand upon it. We want to be able to distinguish between different types of ships, and we want to use freely-available satellite imagery.
|
||||
|
||||
|
||||
To that end, we'll be using [this dataset](https://universe.roboflow.com/ibl-huczk/ships-2fvbx), which contains 3400 labeled images taken from Sentinel-2 (10m/px) and PlanetScope (3m/px) satellites. Ships in these images are labeled by drawing an outline around them:
|
||||
|
||||
|
||||
|
||||
|
||||
The image above shows three ships and what is known as an STS-- a "Ship-To-Ship" transfer-- which is when a ship is transferring cargo to another ship. There are a total of seven classes of ship in this dataset:
|
||||
|
||||
The image above shows three ships and what is known as an STS -- a "Ship-To-Ship" transfer -- which is when a ship is transferring cargo to another ship. There are a total of seven classes of ship in this dataset:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
This dataset can be downloaded directly from Roboflow using the following code:
|
||||
|
||||
|
||||
```python
|
||||
from roboflow import Roboflow
|
||||
rf = Roboflow(api_key="<YOUR API KEY>")
|
||||
project = rf.workspace('ibl-huczk').project("ships-2fvbx")
|
||||
dataset = project.version("1").download("yolov5")
|
||||
```
|
||||
You'll need to get your own API key from Roboflow, which you can do [here](https://app.roboflow.com/account/api), and insert it in the second line of code. Roboflow is a platform for managing and training deep learning models on custom datasets. It's free to use for up to 3 projects, and hosts a large number of datasets that you can use to train your models. To use a different dataset, you can simply change the project name and version number in the second and third lines of code.
|
||||
You'll need to get your own API key from Roboflow, which you can do [here](https://app.roboflow.com/account/api), and insert it in the second line of code. Roboflow is a platform for managing and training deep learning models on custom datasets. It's free to use for up to three projects, and hosts a large number of datasets that you can use to train your models. To use a different dataset, you can simply change the project name and version number in the second and third lines of code.
|
||||
|
||||
|
||||
Finally, we can train our YOLOv5 model on the dataset we just downloaded in just one line of code:
|
||||
|
||||
|
||||
```python
|
||||
!python train.py --data {dataset.location}/data.yaml --batch 32 --cache
|
||||
```
|
||||
|
||||
|
||||
This should take about an hour.
|
||||
|
||||
|
||||
### Accuracy Assessment
|
||||
|
||||
|
||||
Using Tensorboard, we can log the performance of our model over the course of the training process:
|
||||
|
||||
|
||||
:::{.column-page}
|
||||
|
||||
|
||||
<iframe src='https://tensorboard.dev/experiment/Yiyl7AsoQcyJ3uw699CR8A/#scalars' width='100%' height='700px'></iframe>
|
||||
|
||||
|
||||
:::
|
||||
|
||||
|
||||
One metric in particular, **mAP 0.5**, is a good indicator of how well our model is performing. We can see it increasing rapidly at first, and then leveling off after around 30 epochs of training. The rest of this subsection will explain what exactly the mAP 0.5 value represents in this context. If you're interested in training your own model at some point, the rest of this subsection will be of interest. If you're just interested in deploying a pre-trained model, you can skip ahead to the next subsection.
|
||||
|
||||
In the past when we've worked on machine learning projects (for example in the makeshift refinery identifion tutorial), our training and validation data was a set of points-- geographic coordinates-- which we labeled as either being a refinery or not. Calculating the accuracy of that model was fairly straightforward, since predictions were either true positives, true negatives, false positives, or false negatives.
|
||||
|
||||
In the past when we've worked on machine learning projects (for example in the makeshift refinery identifion tutorial), our training and validation data was a set of points -- geographic coordinates -- which we labeled as either being a refinery or not. Calculating the accuracy of that model was fairly straightforward, since predictions were either true positives, true negatives, false positives or false negatives.
|
||||
|
||||
|
||||
This is slightly more complicated for object detection. We're not going pixel-by-pixel and trying to say "this is a ship" or "this is not a ship." Instead, we're looking at a larger image, and trying to draw boxes around the ships. The problem is that there are many ways to draw a box around a ship. The image below shows the labels used in our training data to indicate the location of ships.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
The predicted bounding boxes are very close to the actual bounding boxes, but they're not exactly the same. The first step in evaluating the performance of our model is to determine how close the predicted boxes are to the actual boxes. We can do this by calculating the **intersection over union** (IoU) of the predicted and actual boxes. This is essentially a measure of how much overlap there is between the the predicted and actual boxes:
|
||||
|
||||
|
||||
|
||||
|
||||
{height=400px}
|
||||
|
||||
|
||||
|
||||
|
||||
The IoU is a value between 0 and 1, where 0 means that the boxes don't overlap at all, and 1 means that the boxes overlap perfectly. Now we can set a threshold value for the IoU, and say that if the IoU is greater than that threshold, then we'll count that as a correct prediction. Now that we can classify a prediction as correct or incorrect, we can calculate two important metrics:
|
||||
$$\text{Precision} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}}$$
|
||||
|
||||
|
||||
This is the proportion of positive identifications that are actually correct. If my model detects 100 ships and 90 of them are actually ships, then my precision is 90%.
|
||||
|
||||
|
||||
$$\text{Recall} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Negatives}}$$
|
||||
|
||||
|
||||
This is the proportion of actual positives that are identified correctly. If there are 100 ships in the image, and my model detects 90 of them, then my recall is 90%.
|
||||
|
||||
|
||||
These two metrics are inversely related; I could easily get 100% recall by drawing lots of boxes everywhere to increase my chances of detecting all the ships. Conversely, I could get 100% precision by being extremely conservative and just drawing one or two boxes around the ships I'm most confident about. The key is to maximize *both*: we want our model to be sensitive enough to detect as many ships as possible (high recall), but also precise enough to only draw boxes around the ships that are actually there (high precision). Researchers find this balance using a **Precision-Recall curve** (PR curve), which plots precision on the y-axis and recall on the x-axis. Below is the Precision-Recall curve for our final model, for each class:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Starting from the top left corner, we set a very high confidence threshold: precision is 1, meaning that every box we draw is a ship, but recall is near 0 meaning that we're not detecting any ships. As we lower the confidence threshold, we start to detect more ships, but we also start to draw boxes around things that aren't ships. Towards the middle of the curve, we're detecting most of the ships, but we're also drawing boxes around a lot of false positives. Towards the bottom right corner, we're detecting all the ships, but we're also generating lots of false positives.
|
||||
|
||||
|
||||
The goal is to find the point on the curve where precision and recall are both high; the closer the peak of our curve is to the top right corner, the better. A perfect model would touch the top right corner: it would have precision of 1 and recall of 1, detecting all of the ships without making any false positives. The area under this curve is called the **Average Precision** (AP), and is a measure of how close the curve is to the top right corner. The perfect model would have an AP of 1.
|
||||
|
||||
Some of classes have a very high AP-- the value for the Aircraft Carrier class is 0.995, which is very high (though this could be down to the fact that we have a relatively small number of images with aircraft carriers in them). Ship-To-Ship (STS) transfer operations also have a high AP, at 0.951. However, other classes-- notably the "Ship" class-- have a low AP. This may be because the "Ship" class is a catch-all for any ship that doesn't fit into one of the other classes, so it encompasses lots of weird looking ships.
|
||||
|
||||
Some classes have a very high AP -- the value for the Aircraft Carrier class is 0.995, which is very high (though this could be down to the fact that we have a relatively small number of images with aircraft carriers in them). Ship-To-Ship (STS) transfer operations also have a high AP, at 0.951. However, other classes -- notably the "Ship" class -- have a low AP. This may be because the "Ship" class is a catch-all for any ship that doesn't fit into one of the other classes, so it encompasses lots of weird looking ships.
|
||||
|
||||
|
||||
Finally, the **mean Average Precision** (mAP) is the average of the AP for each class, shown as the thick blue line above. Remember, all of this is premised on using a 0.5 threshold in the overlap (IoU) between our predicted boxes and the labels, which is why the final metric is called **mAP 0.5**. The mAP 0.5 for our model is 0.775, which is pretty good.
|
||||
|
||||
|
||||
This number is very useful when training a model in several different ways using the same dataset, in order to select the best performing one. It's not that useful for comparing models trained on different datasets, since the mAP 0.5 is dependent on the number of classes in the dataset and the nature of those classes. For example, in the next section we'll be using a different model trained on the DOTA dataset which has a mAP 0.5 of around 0.68, largely due to the fact that it has around twice as many classes and many of them are similar to each other.
|
||||
|
||||
|
||||
|
||||
|
||||
## Inference
|
||||
|
||||
|
||||
Now that we've got a trained model, we can use it to conduct object detection on new images. we'll build a data processing pipeline in three steps by:
|
||||
|
||||
|
||||
1. Loading our trained model
|
||||
2. Creating an interactive map to define the area we want to analyze.
|
||||
3. Defining a function to run object detection within this area.
|
||||
|
||||
|
||||
### 1. Loading a trained model
|
||||
|
||||
|
||||
During the training process, YOLO is iteratively tweaking the model to try to maximize mAP 0.5. It automatically saves the best version of the model in the following location: `YOLOv5_RS/runs/train/exp/weights/best.pt`. You can save this file for later use, which I have done in case you just want to use this model without having to train it yourself. I've also included several other pre-trained models which you can find in the `YOLOv5_RS/weights/` directory, including:
|
||||
|
||||
|
||||
* `lowres_ships.pt`: the model we just trained on Sentinel-2 imagery.
|
||||
|
||||
|
||||
* `aircraft.pt`: trained on the high resolution [Airbus Aircraft Detection Dataset](https://www.kaggle.com/datasets/airbusgeo/airbus-aircrafts-sample-dataset).
|
||||
|
||||
|
||||
* `general.pt`: trained on the [DOTA dataset](https://captain-whu.github.io/DOTA/dataset.html) by [Kevin Guo](https://github.com/KevinMuyaoGuo/yolov5s_for_satellite_imagery#readme). This model works great on high resolution satellite imagery, and can detect the following classes: plane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, large vehicle, small vehicle, helicopter, roundabout, soccer field, swimming pool, container crane, airport and helipad.
|
||||
|
||||
|
||||
So far, we've trained a model to detect ships in Sentinel-2 imagery. But to show the versatility of this general approach, the rest of this tutorial will load up the `general.pt` model, and use it to detect a wide range of aircraft in high resolution imagery.
|
||||
|
||||
|
||||
### 2. Loading the input imagery
|
||||
|
||||
|
||||
To get started with object detection on satellite imagery using these pre-trained models, we need to define an Area of Interest (AOI) and load satellite imagery. We'll do this by accessing Google Earth Engine from the Python notebook we're working in, and creating an interactive map that will let us draw an AOI for analysis.
|
||||
|
||||
|
||||
First, we first need to import a few packages:
|
||||
|
||||
|
||||
```python
|
||||
!pip install geemap -q
|
||||
import pandas as pd
|
||||
@@ -173,49 +246,63 @@ import torch
|
||||
import PIL
|
||||
```
|
||||
|
||||
|
||||
Once we've done this, we'll also need to log in to Google Earth Engine using its Python API in order to access the satellite imagery. Running these two lines of code will generate a prompt with instructions; you have to click the link, confirm that you give the notebook permission to access your Earth Engine account, and paste the authentication code in the provided dialogue box.
|
||||
|
||||
|
||||
```python
|
||||
ee.Authenticate()
|
||||
ee.Initialize()
|
||||
```
|
||||
|
||||
Great-- now we can load high resolution imagery from the National Agriculture Imagery Program (NAIP) and create an interactive map. For this example, I'm centering the map on the [Davis-Monthan Airplane Boneyard](https://en.wikipedia.org/wiki/309th_Aerospace_Maintenance_and_Regeneration_Group). This is where the airforce retires and restores aircraft, so it will have lots of airplanes of different kinds for us to identify.
|
||||
|
||||
Great, now we can load high resolution imagery from the National Agriculture Imagery Program (NAIP) and create an interactive map. For this example, I'm centering the map on the [Davis-Monthan Airplane Boneyard](https://en.wikipedia.org/wiki/309th_Aerospace_Maintenance_and_Regeneration_Group). This is where the US Air force retires and restores aircraft, so it will have lots of airplanes of different kinds for us to identify.
|
||||
|
||||
|
||||
First, we want to define a function called `detect` that will accept four arguments:
|
||||
|
||||
|
||||
* `input`: the satellite imagery we want to analyze.
|
||||
* `visParams`: a dictionary of visualization parameters for the imagery.
|
||||
* `weight`: the name of the pre-trained model we want to use.
|
||||
* `labels`: a boolean indicating whether we want to display the labels on the processed image.
|
||||
|
||||
|
||||
|
||||
|
||||
```python
|
||||
def detect(input, visParams, weight, labels=True):
|
||||
|
||||
|
||||
# Get the AOI from the map
|
||||
aoi = ee.FeatureCollection(Map.draw_features)
|
||||
mapScale=Map.getScale()
|
||||
|
||||
|
||||
# Visualize the raster in Earth Engine and get a download URL
|
||||
image_url=input.visualize(bands=visParams['bands'], max=visParams['max']).getThumbURL({"region":aoi.geometry(), 'scale':mapScale})
|
||||
|
||||
|
||||
# Load the image into a PIL image
|
||||
response = requests.get(image_url)
|
||||
img = Image.open(BytesIO(response.content))
|
||||
|
||||
|
||||
# Load the model
|
||||
model =torch.hub.load('.','custom', path='weights/{}.pt'.format(weight),source='local',_verbose=False)
|
||||
|
||||
# Run inference
|
||||
results = model(img)
|
||||
|
||||
|
||||
# Count the number of detections
|
||||
counts=pd.DataFrame(results.pandas().xyxy[0].groupby('name').size()).reset_index().rename(columns={0:'count','name':'detected'}).set_index('count')
|
||||
|
||||
|
||||
# Display the results
|
||||
results.show(labels=labels)
|
||||
|
||||
|
||||
# Print the number of detections and the date of the image
|
||||
print(ee.Date(input.get('system:time_start')).format("dd-MM-yyyy").getInfo())
|
||||
print(counts)
|
||||
@@ -223,12 +310,15 @@ def detect(input, visParams, weight, labels=True):
|
||||
return counts
|
||||
```
|
||||
|
||||
|
||||
Now, we can load the NAIP imagery and create an interactive map.
|
||||
|
||||
|
||||
```python
|
||||
# load the past 10 years of NAIP imagery
|
||||
naip = ee.ImageCollection('USDA/NAIP/DOQQ').filter(ee.Filter.date('2012-01-01', '2022-01-01'))
|
||||
|
||||
|
||||
# set some thresholds
|
||||
trueColorVis = {
|
||||
'bands':['R', 'G', 'B'],
|
||||
@@ -236,6 +326,7 @@ trueColorVis = {
|
||||
'max': 300,
|
||||
};
|
||||
|
||||
|
||||
# initialize our map
|
||||
Map = geemap.Map()
|
||||
Map.setCenter(-110.84,32.16,17)
|
||||
@@ -243,43 +334,61 @@ Map.addLayer(naip.first(), trueColorVis, "naip")
|
||||
Map
|
||||
```
|
||||
|
||||
|
||||
This will generate a small map with some drawing tools on the left side. We can use these tools to draw a polygon around the area we want to analyze. Use the drawing tools to draw a rectangle around an area of interest.
|
||||
|
||||
|
||||
Finally, we can run the detection on the imagery. We'll do this by iterating through the collection of images, and running the `detect` function on each one. We'll also store the results in a dataframe so we can analyze them later.
|
||||
|
||||
|
||||
``` python
|
||||
# Get the polygon we just drew on the map
|
||||
aoi=ee.FeatureCollection(Map.draw_features)
|
||||
|
||||
|
||||
# Get a list of all the images in the collection
|
||||
naip_list=naip.filterBounds(aoi).toList(naip.size())
|
||||
|
||||
|
||||
# Iterate through the list of images and run detection on each one
|
||||
for num in range(0,(img_list.size()).getInfo()):
|
||||
detect(ee.Image(naip_list.get(num)), trueColorVis,'general',labels=False)
|
||||
df=df.append(detection) # store the results in a dataframe
|
||||
```
|
||||
|
||||
|
||||
Below is the result of the detection on the latest image in the collection:
|
||||
|
||||
|
||||
:::{.column-screen}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
:::
|
||||
|
||||
This image shows a remarkable degree of accuracy being achieved by our model. Inference took just 822.2 milliseconds, and it seems to be doing pretty well. The model identifies over 100 different kinds of aircraft (orange boxes) of many shapes and sizes, civilian and military, without missing a single one. It also identifies around 20 different types of helicopter (blue boxes) in the top right and even spots the cars on the highway and in the parking lots (red boxes). It's not perfect-- it thinks there's a ship in the bottom left corner near the shed (yellow box); in reality this appears to be half of a plane's fuselage, an understandable mistake given how long it took *me* to figure out what it was.
|
||||
|
||||
This image shows a remarkable degree of accuracy being achieved by our model. Inference took just 822.2 milliseconds, and it seems to be doing pretty well. The model identifies over 100 different kinds of aircraft (orange boxes) of many shapes and sizes, civilian and military, without missing a single one. It also identifies around 20 different types of helicopter (blue boxes) in the top right and even spots the cars on the highway and in the parking lots (red boxes). It's not perfect -- it thinks there's a ship in the bottom left corner near the shed (yellow box); in reality this appears to be half of a plane's fuselage, an understandable mistake given how long it took *me* to figure out what it was.
|
||||
|
||||
|
||||
|
||||
|
||||
<!--
|
||||
Even through we trained our model on Sentinel-2 imagery (10 meters per pixel), it can still be used on imagery from different satellites as long as they have a broadly similar resolution. A ship in PlanetScope imagery (3 meters per pixel) will look roughly similar to a ship in Sentinel-2 imagery. Using PlanetScope has another big over Sentinel-2 beyond its higher spatial resolution: it has a much higher revisit rate (daily instead of 5 days). Though *downloading* PlanetScope imagery isn't free, you *can* generate a timelapse image of any area on Earth using Planet's [Planet Stories](https://www.planet.com/stories/create) tool. Simply create a free account and follow the instructions to generate a timelapse of an area of interest. You can then download the timelapse video and use it as input to our model.
|
||||
Even though we trained our model on Sentinel-2 imagery (10 meters per pixel), it can still be used on imagery from different satellites as long as they have a broadly similar resolution. A ship in PlanetScope imagery (3 meters per pixel) will look roughly similar to a ship in Sentinel-2 imagery. Using PlanetScope has another big advantage over Sentinel-2 beyond its higher spatial resolution: it has a much higher revisit rate (daily instead of 5 days). Though *downloading* PlanetScope imagery isn't free, you *can* generate a timelapse image of any area on Earth using Planet's [Planet Stories](https://www.planet.com/stories/create) tool. Simply create a free account and follow the instructions to generate a timelapse of an area of interest. You can then download the timelapse video and use it as input to our model.
|
||||
|
||||
|
||||
Once you've done this, you can run the following line of code to automatically identify ships in the timelapse video:
|
||||
|
||||
|
||||
|
||||
|
||||
{{< video images/mikolayiv.mp4 >}}
|
||||
|
||||
|
||||
|
||||
|
||||
-->
|
||||
|
||||
|
||||
|
||||
|
||||
-->
|
||||
@@ -1,6 +1,6 @@
|
||||
project:
|
||||
type: book
|
||||
output-dir: docs
|
||||
output-dir: ../docs
|
||||
|
||||
|
||||
book:
|
||||
@@ -11,34 +11,33 @@ book:
|
||||
- part: "A. Introduction"
|
||||
chapters:
|
||||
- index.qmd
|
||||
- ch1.qmd
|
||||
- ch2.qmd
|
||||
- A2_Remote_Sensing.qmd
|
||||
- A3_Data_Acquisition.qmd
|
||||
- part: "B. Google Earth Engine"
|
||||
chapters:
|
||||
- F1.qmd
|
||||
- F2.qmd
|
||||
- F4.qmd
|
||||
- F5.qmd
|
||||
- B1_Getting_Started.qmd
|
||||
- B2_Interpreting_Images.qmd
|
||||
- B3_Image_Series.qmd
|
||||
- B4_Vectors_Tables.qmd
|
||||
- part: "C. Case Studies"
|
||||
chapters:
|
||||
- lights.qmd
|
||||
- refineries.qmd
|
||||
- ships.qmd
|
||||
- blast.qmd
|
||||
- object_detection.qmd
|
||||
repo-url: https://github.com/oballinger/GEE_OSINT/
|
||||
- C1_Lights.qmd
|
||||
- C2_Refineries.qmd
|
||||
- C3_Blast.qmd
|
||||
- C4_Ships.qmd
|
||||
- C5_Object_Detection.qmd
|
||||
repo-url: https://github.com/oballinger/RS4OSINT/
|
||||
google-analytics: G-RK9ZLZQ6GL
|
||||
repo-actions: [edit]
|
||||
downloads: [pdf, epub]
|
||||
sharing: [twitter, facebook]
|
||||
favicon: favicon.ico
|
||||
favicon: ../favicon.ico
|
||||
sidebar:
|
||||
logo: logo_white.png
|
||||
logo: ../images/logo_white.png
|
||||
collapse-level: 1
|
||||
|
||||
|
||||
|
||||
bibliography: references.bib
|
||||
|
||||
format:
|
||||
html:
|
||||
@@ -47,8 +46,9 @@ format:
|
||||
light: cosmo
|
||||
code-copy: true
|
||||
code-overflow: wrap
|
||||
highlight-style: monokai.theme
|
||||
linkcolor: "#34a832"
|
||||
highlight-style: ../monokai.theme
|
||||
linkcolor: "#34a832"
|
||||
number-sections: false
|
||||
|
||||
pdf:
|
||||
documentclass: scrreprt
|
||||
@@ -57,4 +57,4 @@ format:
|
||||
- \@addtoreset{chapter}{part}
|
||||
- \makeatother
|
||||
epub:
|
||||
cover-image: cover.png
|
||||
cover-image: ../images/logo_black.png
|
||||
233
chapters/index.log
Normal file
233
chapters/index.log
Normal file
@@ -0,0 +1,233 @@
|
||||
This is XeTeX, Version 3.141592653-2.6-0.999994 (TeX Live 2022) (preloaded format=xelatex 2022.10.26) 17 APR 2023 11:17
|
||||
entering extended mode
|
||||
restricted \write18 enabled.
|
||||
%&-line parsing enabled.
|
||||
**index.tex
|
||||
(./index.tex
|
||||
LaTeX2e <2022-06-01> patch level 5
|
||||
L3 programming layer <2022-09-28> (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrreprt.cls
|
||||
Document Class: scrreprt 2022/10/12 v3.38 KOMA-Script document class (report)
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrkbase.sty
|
||||
Package: scrkbase 2022/10/12 v3.38 KOMA-Script package (KOMA-Script-dependent basics and keyval usage)
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrbase.sty
|
||||
Package: scrbase 2022/10/12 v3.38 KOMA-Script package (KOMA-Script-independent basics and keyval usage)
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrlfile.sty
|
||||
Package: scrlfile 2022/10/12 v3.38 KOMA-Script package (file load hooks)
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrlfile-hook.sty
|
||||
Package: scrlfile-hook 2022/10/12 v3.38 KOMA-Script package (using LaTeX hooks)
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrlogo.sty
|
||||
Package: scrlogo 2022/10/12 v3.38 KOMA-Script package (logo)
|
||||
))) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/graphics/keyval.sty
|
||||
Package: keyval 2022/05/29 v1.15 key=value parser (DPC)
|
||||
\KV@toks@=\toks16
|
||||
)
|
||||
Applying: [2021/05/01] Usage of raw or classic option list on input line 252.
|
||||
Already applied: [0000/00/00] Usage of raw or classic option list on input line 368.
|
||||
)) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/tocbasic.sty
|
||||
Package: tocbasic 2022/10/12 v3.38 KOMA-Script package (handling toc-files)
|
||||
\scr@dte@tocline@numberwidth=\skip47
|
||||
\scr@dte@tocline@numbox=\box51
|
||||
)
|
||||
Package tocbasic Info: omitting babel extension for `toc'
|
||||
(tocbasic) because of feature `nobabel' available
|
||||
(tocbasic) for `toc' on input line 137.
|
||||
Class scrreprt Info: You've used standard option `oneside'.
|
||||
(scrreprt) This is correct!
|
||||
(scrreprt) Internally I'm using `twoside=false'.
|
||||
(scrreprt) If you'd like to set the option with \KOMAoptions,
|
||||
(scrreprt) you'd have to use `twoside=false' there
|
||||
(scrreprt) instead of `oneside', too.
|
||||
Class scrreprt Info: File `scrsize11pt.clo' used instead of
|
||||
(scrreprt) file `scrsize11.clo' to setup font sizes on input line 2614.
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/scrsize11pt.clo
|
||||
File: scrsize11pt.clo 2022/10/12 v3.38 KOMA-Script font size class option (11pt)
|
||||
) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/koma-script/typearea.sty
|
||||
Package: typearea 2022/10/12 v3.38 KOMA-Script package (type area)
|
||||
\ta@bcor=\skip48
|
||||
\ta@div=\count181
|
||||
Package typearea Info: You've used standard option `letterpaper'.
|
||||
(typearea) This is correct!
|
||||
(typearea) Internally I'm using `paper=letter'.
|
||||
(typearea) If you'd like to set the option with \KOMAoptions,
|
||||
(typearea) you'd have to use `paper=letter' there
|
||||
(typearea) instead of `letterpaper', too.
|
||||
Package typearea Info: You've used standard option `oneside'.
|
||||
(typearea) This is correct!
|
||||
(typearea) Internally I'm using `twoside=false'.
|
||||
(typearea) If you'd like to set the option with \KOMAoptions,
|
||||
(typearea) you'd have to use `twoside=false' there
|
||||
(typearea) instead of `oneside', too.
|
||||
\ta@hblk=\skip49
|
||||
\ta@vblk=\skip50
|
||||
\ta@temp=\skip51
|
||||
\footheight=\skip52
|
||||
Package typearea Info: These are the values describing the layout:
|
||||
(typearea) DIV = 11
|
||||
(typearea) BCOR = 0.0pt
|
||||
(typearea) \paperwidth = 614.295pt
|
||||
(typearea) \textwidth = 446.76004pt
|
||||
(typearea) DIV departure = -14%
|
||||
(typearea) \evensidemargin = 11.49748pt
|
||||
(typearea) \oddsidemargin = 11.49748pt
|
||||
(typearea) \paperheight = 794.96999pt
|
||||
(typearea) \textheight = 582.20026pt
|
||||
(typearea) \topmargin = -37.40001pt
|
||||
(typearea) \headheight = 17.0pt
|
||||
(typearea) \headsep = 20.40001pt
|
||||
(typearea) \topskip = 11.0pt
|
||||
(typearea) \footskip = 47.6pt
|
||||
(typearea) \baselineskip = 13.6pt
|
||||
(typearea) on input line 1767.
|
||||
)
|
||||
\c@part=\count182
|
||||
\c@chapter=\count183
|
||||
\c@section=\count184
|
||||
\c@subsection=\count185
|
||||
\c@subsubsection=\count186
|
||||
\c@paragraph=\count187
|
||||
\c@subparagraph=\count188
|
||||
\scr@dte@chapter@maxnumwidth=\skip53
|
||||
Class scrreprt Info: using compatibility default `afterindent=bysign'
|
||||
(scrreprt) for `\chapter on input line 5902.
|
||||
\scr@dte@section@maxnumwidth=\skip54
|
||||
Class scrreprt Info: using compatibility default `runin=bysign'
|
||||
(scrreprt) for `\section on input line 5913.
|
||||
Class scrreprt Info: using compatibility default `afterindent=bysign'
|
||||
(scrreprt) for `\section on input line 5913.
|
||||
\scr@dte@part@maxnumwidth=\skip55
|
||||
Class scrreprt Info: using compatibility default `afterindent=true'
|
||||
(scrreprt) for `\part on input line 5922.
|
||||
\scr@dte@subsection@maxnumwidth=\skip56
|
||||
Class scrreprt Info: using compatibility default `runin=bysign'
|
||||
(scrreprt) for `\subsection on input line 5932.
|
||||
Class scrreprt Info: using compatibility default `afterindent=bysign'
|
||||
(scrreprt) for `\subsection on input line 5932.
|
||||
\scr@dte@subsubsection@maxnumwidth=\skip57
|
||||
Class scrreprt Info: using compatibility default `runin=bysign'
|
||||
(scrreprt) for `\subsubsection on input line 5942.
|
||||
Class scrreprt Info: using compatibility default `afterindent=bysign'
|
||||
(scrreprt) for `\subsubsection on input line 5942.
|
||||
\scr@dte@paragraph@maxnumwidth=\skip58
|
||||
Class scrreprt Info: using compatibility default `runin=bysign'
|
||||
(scrreprt) for `\paragraph on input line 5953.
|
||||
Class scrreprt Info: using compatibility default `afterindent=bysign'
|
||||
(scrreprt) for `\paragraph on input line 5953.
|
||||
\scr@dte@subparagraph@maxnumwidth=\skip59
|
||||
Class scrreprt Info: using compatibility default `runin=bysign'
|
||||
(scrreprt) for `\subparagraph on input line 5963.
|
||||
Class scrreprt Info: using compatibility default `afterindent=bysign'
|
||||
(scrreprt) for `\subparagraph on input line 5963.
|
||||
\abovecaptionskip=\skip60
|
||||
\belowcaptionskip=\skip61
|
||||
\c@pti@nb@sid@b@x=\box52
|
||||
Package tocbasic Info: omitting babel extension for `lof'
|
||||
(tocbasic) because of feature `nobabel' available
|
||||
(tocbasic) for `lof' on input line 7140.
|
||||
\scr@dte@figure@maxnumwidth=\skip62
|
||||
\c@figure=\count189
|
||||
Package tocbasic Info: omitting babel extension for `lot'
|
||||
(tocbasic) because of feature `nobabel' available
|
||||
(tocbasic) for `lot' on input line 7157.
|
||||
\scr@dte@table@maxnumwidth=\skip63
|
||||
\c@table=\count190
|
||||
Class scrreprt Info: Redefining `\numberline' on input line 7328.
|
||||
\bibindent=\dimen138
|
||||
) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsmath/amsmath.sty
|
||||
Package: amsmath 2022/04/08 v2.17n AMS math features
|
||||
\@mathmargin=\skip64
|
||||
For additional information on amsmath, use the `?' option.
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsmath/amstext.sty
|
||||
Package: amstext 2021/08/26 v2.01 AMS text
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsmath/amsgen.sty
|
||||
File: amsgen.sty 1999/11/30 v2.0 generic functions
|
||||
\@emptytoks=\toks17
|
||||
\ex@=\dimen139
|
||||
)) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsmath/amsbsy.sty
|
||||
Package: amsbsy 1999/11/29 v1.2d Bold Symbols
|
||||
\pmbraise@=\dimen140
|
||||
) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsmath/amsopn.sty
|
||||
Package: amsopn 2022/04/08 v2.04 operator names
|
||||
)
|
||||
\inf@bad=\count191
|
||||
LaTeX Info: Redefining \frac on input line 234.
|
||||
\uproot@=\count192
|
||||
\leftroot@=\count193
|
||||
LaTeX Info: Redefining \overline on input line 399.
|
||||
LaTeX Info: Redefining \colon on input line 410.
|
||||
\classnum@=\count194
|
||||
\DOTSCASE@=\count195
|
||||
LaTeX Info: Redefining \ldots on input line 496.
|
||||
LaTeX Info: Redefining \dots on input line 499.
|
||||
LaTeX Info: Redefining \cdots on input line 620.
|
||||
\Mathstrutbox@=\box53
|
||||
\strutbox@=\box54
|
||||
LaTeX Info: Redefining \big on input line 722.
|
||||
LaTeX Info: Redefining \Big on input line 723.
|
||||
LaTeX Info: Redefining \bigg on input line 724.
|
||||
LaTeX Info: Redefining \Bigg on input line 725.
|
||||
\big@size=\dimen141
|
||||
LaTeX Font Info: Redeclaring font encoding OML on input line 743.
|
||||
LaTeX Font Info: Redeclaring font encoding OMS on input line 744.
|
||||
\macc@depth=\count196
|
||||
LaTeX Info: Redefining \bmod on input line 905.
|
||||
LaTeX Info: Redefining \pmod on input line 910.
|
||||
LaTeX Info: Redefining \smash on input line 940.
|
||||
LaTeX Info: Redefining \relbar on input line 970.
|
||||
LaTeX Info: Redefining \Relbar on input line 971.
|
||||
\c@MaxMatrixCols=\count197
|
||||
\dotsspace@=\muskip16
|
||||
\c@parentequation=\count198
|
||||
\dspbrk@lvl=\count199
|
||||
\tag@help=\toks18
|
||||
\row@=\count266
|
||||
\column@=\count267
|
||||
\maxfields@=\count268
|
||||
\andhelp@=\toks19
|
||||
\eqnshift@=\dimen142
|
||||
\alignsep@=\dimen143
|
||||
\tagshift@=\dimen144
|
||||
\tagwidth@=\dimen145
|
||||
\totwidth@=\dimen146
|
||||
\lineht@=\dimen147
|
||||
\@envbody=\toks20
|
||||
\multlinegap=\skip65
|
||||
\multlinetaggap=\skip66
|
||||
\mathdisplay@stack=\toks21
|
||||
LaTeX Info: Redefining \[ on input line 2953.
|
||||
LaTeX Info: Redefining \] on input line 2954.
|
||||
) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsfonts/amssymb.sty
|
||||
Package: amssymb 2013/01/14 v3.01 AMS font symbols
|
||||
(/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/amsfonts/amsfonts.sty
|
||||
Package: amsfonts 2013/01/14 v3.01 Basic AMSFonts support
|
||||
\symAMSa=\mathgroup4
|
||||
\symAMSb=\mathgroup5
|
||||
LaTeX Font Info: Redeclaring math symbol \hbar on input line 98.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathfrak' in version `bold'
|
||||
(Font) U/euf/m/n --> U/euf/b/n on input line 106.
|
||||
)) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/generic/iftex/iftex.sty
|
||||
Package: iftex 2022/02/03 v1.0f TeX engine tests
|
||||
) (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/unicode-math/unicode-math.sty (/Users/ollieballinger/Library/TinyTeX/texmf-dist/tex/latex/l3kernel/expl3.sty
|
||||
Package: expl3 2023-02-07 L3 programming layer (loader)
|
||||
|
||||
! LaTeX Error: Mismatched LaTeX support files detected.
|
||||
(LaTeX) Loading 'expl3.sty' aborted!
|
||||
(LaTeX)
|
||||
(LaTeX) The L3 programming layer in the LaTeX format
|
||||
(LaTeX) is dated 2022-09-28, but in your TeX tree the files require
|
||||
(LaTeX) at least 2023-02-07.
|
||||
|
||||
For immediate help type H <return>.
|
||||
...
|
||||
|
||||
l.77 \ExplLoaderFileDate{expl3.sty}}
|
||||
%
|
||||
Here is how much of TeX's memory you used:
|
||||
4567 strings out of 477723
|
||||
94617 string characters out of 5841548
|
||||
564227 words of memory out of 5000000
|
||||
25658 multiletter control sequences out of 15000+600000
|
||||
469267 words of font info for 29 fonts, out of 8000000 for 9000
|
||||
14 hyphenation exceptions out of 8191
|
||||
108i,1n,106p,10600b,270s stack positions out of 10000i,1000n,20000p,200000b,200000s
|
||||
|
||||
No pages of output.
|
||||
@@ -1,6 +1,6 @@
|
||||
# Overview {.unnumbered}
|
||||
# Overview
|
||||
|
||||
The analysis of satellite imagery is a foundational element of open source investigations. In the past decade, the quantity, quality, and availability thereof has increased dramatically. Capabilities and insights that were once only available to governments are now accessible to the general public. Satellite imagery is being used to collect evidence of genocide and other war crimes in [Ukraine](https://www.nbcnews.com/science/science-news/ukraine-satellites-war-crimes-rcna26291), [Nigeria](https://www.amnesty.org/en/latest/news/2016/04/nigeria-military-cover-up-of-mass-slaughter-at-zaria-exposed/), [Burundi](https://www.amnesty.org/en/latest/news/2016/01/burundi-satellite-evidence-supports-witness-accounts-of-mass-graves/), [Cameroon](https://www.amnesty.org/en/latest/news/2021/07/cameroon-satellite-images-reveal-devastation-in-anglophone-regions-2/), [the DRC](https://www.aaas.org/resources/satellite-imagery-assessment-forced-relocations-near-luiswishi-mine), [South Sudan](https://gsp.yale.edu/case-studies/sudan/maps-satellite-images/other-darfur-satellite-imagery), [Papua](https://gsp.yale.edu/resources/maps-satellite-images/papua), and [Venezuela](https://www.hrw.org/report/2016/04/04/unchecked-power/police-and-military-raids-low-income-and-immigrant-communities). It has been used to [monitor environmental degradation](https://www.theguardian.com/environment/2016/mar/02/new-satellite-mapping-a-game-changer-against-illegal-logging) and hold extractive industries to account from [Iraq](https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/) to [Guatemala](https://www.planet.com/pulse/the-observatory-of-extractive-industries-oie-shines-a-light-on-the-mining-industry-using-planets-satellite-data/). The ability to analyze satellite imagery is a critical skill for anyone interested in open source investigations.
|
||||
The analysis of satellite imagery is a foundational element of open source investigations. Its quality, quantity and availability has increased dramatically in the past decade. Capabilities and insights that were once only available to governments are now accessible to the general public. Satellite imagery is being used to collect evidence of genocide and other war crimes in [Ukraine](https://www.nbcnews.com/science/science-news/ukraine-satellites-war-crimes-rcna26291), [Nigeria](https://www.amnesty.org/en/latest/news/2016/04/nigeria-military-cover-up-of-mass-slaughter-at-zaria-exposed/), [Burundi](https://www.amnesty.org/en/latest/news/2016/01/burundi-satellite-evidence-supports-witness-accounts-of-mass-graves/), [Cameroon](https://www.amnesty.org/en/latest/news/2021/07/cameroon-satellite-images-reveal-devastation-in-anglophone-regions-2/), [the DRC](https://www.aaas.org/resources/satellite-imagery-assessment-forced-relocations-near-luiswishi-mine), [South Sudan](https://gsp.yale.edu/case-studies/sudan/maps-satellite-images/other-darfur-satellite-imagery), [Papua](https://gsp.yale.edu/resources/maps-satellite-images/papua), and [Venezuela](https://www.hrw.org/report/2016/04/04/unchecked-power/police-and-military-raids-low-income-and-immigrant-communities). It has been used to [monitor environmental degradation](https://www.theguardian.com/environment/2016/mar/02/new-satellite-mapping-a-game-changer-against-illegal-logging) and hold extractive industries to account from [Iraq](https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/) to [Guatemala](https://www.planet.com/pulse/the-observatory-of-extractive-industries-oie-shines-a-light-on-the-mining-industry-using-planets-satellite-data/). The ability to analyze satellite imagery is a critical skill for anyone interested in open source investigations.
|
||||
|
||||
Though no-code platforms such as Sentinelhub have been invaluable in allowing the OSINT community to access and process satellite imagery, the analytical capabilities of these platforms are limited. [Google Earth Engine (GEE)](https://earthengine.google.com/#intro) is a cloud-based platform that stores petabytes of satellite imagery from a variety of sources and allows users to perform advanced analyses on Google servers for free using a browser-based interface. This textbook is designed for investigators who want to perform more sophisticated analysis using geospatial data, and assumes no prior knowledge of coding or remote sensing (satellite imagery analysis). It is organized into two parts: an introduction to remote sensing and GEE, and a series of case studies that demonstrate how to use GEE for open source investigations.
|
||||
|
||||
@@ -10,26 +10,26 @@ As geospatial datasets—particularly satellite imagery collections—increase i
|
||||
|
||||
GEE is free and allows users to write open-source code that can be run by others in one click, thereby yielding fully reproducible results. These features have put GEE on the cutting edge of scientific research. The following plot visualizes the number of journal articles conducted using different geospatial analysis software platforms:
|
||||
|
||||
|
||||
|
||||
|
||||
Despite only being released in 2015, the number of geospatial journal articles using Google Earth Engine (shown in red above) has outpaced every other major geospatial analysis software, including ArcGIS, Python, and R in just five years. GEE applications have been developed and used to present interactive geospatial data visualizations by NGOs, Universities, the United Nations, and the European Commission. By storing and running computations on google servers, GEE is far more accessible to those who don’t have significant local computational resources; all you need is an internet connection.
|
||||
Despite only being released in 2015, the number of geospatial journal articles using Google Earth Engine (shown in red above) has outpaced every other major geospatial analysis software, including ArcGIS, Python, and R in just five years. GEE applications have been developed and used to present interactive geospatial data visualizations by NGOs, Universities, the United Nations, and the European Commission. By storing and running computations on Google servers, GEE is far more accessible to those who don’t have significant local computational resources; all you need is an internet connection.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
A) **Introduction**
|
||||
* Two introductory chapters that provide an overview of remote sensing the different types of satellite imagery available on Google Earth Engine.
|
||||
* [Remote Sensing](ch1.qmd)
|
||||
* [Data Acquisition](ch2.qmd)
|
||||
* [Remote Sensing](chapters/A2_Remote_Sensing.qmd)
|
||||
* [Data Acquisition](chapters/A3_Data_Acquisition.qmd)
|
||||
B) **Google Earth Engine**
|
||||
* Recently, a team of over 100 scientists came together to write a book called ["Cloud-Based Remote Sensing with Google Earth Engine: Fundamentals and Applications"](https://www.eefabook.org/). It's a great resource for learning about remote sensing and Earth Engine. The material in this section is a subset of the book, edited to fit the scope of this guide. If you're interested in learning more, check out the full book.
|
||||
* [Getting Started](F1.qmd)
|
||||
* [Interpreting Images](F2.qmd)
|
||||
* [Image Series](F4.qmd)
|
||||
* [Vectors and Tables](F5.qmd)
|
||||
* [Getting Started](chapters/B1_Getting_Started.qmd)
|
||||
* [Interpreting Images](chapters/B2_Interpreting_Images.qmd)
|
||||
* [Image Series](chapters/B3_Image_Series.qmd)
|
||||
* [Vectors and Tables](chapters/B4_Vectors_Tables.qmd)
|
||||
C) **Case Studies**
|
||||
* A series of case studies that demonstrate how to use Google Earth Engine for open source investigations. Each case study includes a brief introduction to the topic, a step-by-step guide to using Google Earth Engine to analyze satellite imagery, and a discussion of the results.
|
||||
* [War at Night](lights.qmd)
|
||||
* [Refinery Identification](refineries.qmd)
|
||||
* [Ship Detection](ships.qmd)
|
||||
* [Blast Damage Assessment](blast.qmd)
|
||||
* [Object Detection](object_detection.qmd)
|
||||
* [War at Night](chapters/C1_Lights.qmd)
|
||||
* [Refinery Identification](chapters/C2_Refineries.qmd)
|
||||
* [Blast Damage Assessment](chapters/C3_Blast.qmd)
|
||||
* [Ship Detection](chapters/C4_Ships.qmd)
|
||||
* [Object Detection](chapters/C5_Object_Detection.qmd)
|
||||
19444
chapters/index.tex
Normal file
19444
chapters/index.tex
Normal file
File diff suppressed because it is too large
Load Diff
731
docs/A2_Remote_Sensing.html
Normal file
731
docs/A2_Remote_Sensing.html
Normal file
@@ -0,0 +1,731 @@
|
||||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta name="generator" content="quarto-1.3.326">
|
||||
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - 2 Remote Sensing</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
div.columns{display: flex; gap: min(4vw, 1.5em);}
|
||||
div.column{flex: auto; overflow-x: auto;}
|
||||
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
||||
ul.task-list{list-style: none;}
|
||||
ul.task-list li input[type="checkbox"] {
|
||||
width: 0.8em;
|
||||
margin: 0 0.8em 0.2em -1em; /* quarto-specific, see https://github.com/quarto-dev/quarto-cli/issues/4556 */
|
||||
vertical-align: middle;
|
||||
}
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./A3_Data_Acquisition.html" rel="next">
|
||||
<link href="./index.html" rel="prev">
|
||||
<link href="./../favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="site_libs/quarto-contrib/videojs/video.min.js"></script>
|
||||
<link href="site_libs/quarto-contrib/videojs/video-js.css" rel="stylesheet">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
"collapse-after": 3,
|
||||
"panel-placement": "start",
|
||||
"type": "textbox",
|
||||
"limit": 20,
|
||||
"language": {
|
||||
"search-no-results-text": "No results",
|
||||
"search-matching-documents-text": "matching documents",
|
||||
"search-copy-link-title": "Copy link to search",
|
||||
"search-hide-matches-text": "Hide additional matches",
|
||||
"search-more-match-text": "more match in this document",
|
||||
"search-more-matches-text": "more matches in this document",
|
||||
"search-clear-button-title": "Clear",
|
||||
"search-detached-cancel-button-title": "Cancel",
|
||||
"search-submit-button-title": "Submit"
|
||||
}
|
||||
}</script>
|
||||
<script async="" src="https://www.googletagmanager.com/gtag/js?id=G-RK9ZLZQ6GL"></script>
|
||||
|
||||
<script type="text/javascript">
|
||||
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</script>
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body class="nav-sidebar floating">
|
||||
|
||||
|
||||
<div id="quarto-search-results"></div>
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav">
|
||||
<div class="container-fluid d-flex">
|
||||
<button type="button" class="quarto-btn-toggle btn" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<i class="bi bi-layout-text-sidebar-reverse"></i>
|
||||
</button>
|
||||
<nav class="quarto-page-breadcrumbs" aria-label="breadcrumb"><ol class="breadcrumb"><li class="breadcrumb-item"><a href="./index.html">A. Introduction</a></li><li class="breadcrumb-item"><a href="./A2_Remote_Sensing.html"><span class="chapter-number">2</span> <span class="chapter-title">Remote Sensing</span></a></li></ol></nav>
|
||||
<a class="flex-grow-1" role="button" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
</a>
|
||||
<button type="button" class="btn quarto-search-button" aria-label="Search" onclick="window.quartoOpenSearch();">
|
||||
<i class="bi bi-search"></i>
|
||||
</button>
|
||||
</div>
|
||||
</nav>
|
||||
</header>
|
||||
<!-- content -->
|
||||
<div id="quarto-content" class="quarto-container page-columns page-rows-contents page-layout-article">
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse collapse-horizontal sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="quarto-navigation-tool px-1" aria-label="Source Code"><i class="bi bi-github"></i></a>
|
||||
<div class="dropdown">
|
||||
<a href="" title="Download" id="quarto-navigation-tool-dropdown-0" class="quarto-navigation-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Download"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="quarto-navigation-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="dropdown">
|
||||
<a href="" title="Share" id="quarto-navigation-tool-dropdown-1" class="quarto-navigation-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Share"><i class="bi bi-share"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="quarto-navigation-tool-dropdown-1">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="https://twitter.com/intent/tweet?url=|url|">
|
||||
<i class="bi bi-bi-twitter pe-1"></i>
|
||||
Twitter
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="https://www.facebook.com/sharer/sharer.php?u=|url|">
|
||||
<i class="bi bi-bi-facebook pe-1"></i>
|
||||
Facebook
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<a href="" class="quarto-color-scheme-toggle quarto-navigation-tool px-1" onclick="window.quartoToggleColorScheme(); return false;" title="Toggle dark mode"><i class="bi"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-2 flex-shrink-0 align-items-center">
|
||||
<div class="sidebar-search">
|
||||
<div id="quarto-search" class="" title="Search"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="sidebar-menu-container">
|
||||
<ul class="list-unstyled mt-1">
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-1" aria-expanded="true">
|
||||
<span class="menu-text">A. Introduction</span></a>
|
||||
<a class="sidebar-item-toggle text-start" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-1" aria-expanded="true" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">1</span> <span class="chapter-title">Overview</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link active">
|
||||
<span class="menu-text"><span class="chapter-number">2</span> <span class="chapter-title">Remote Sensing</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">3</span> <span class="chapter-title">Data Acquisition</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-2" aria-expanded="false">
|
||||
<span class="menu-text">B. Google Earth Engine</span></a>
|
||||
<a class="sidebar-item-toggle text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-2" aria-expanded="false" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B1_Getting_Started.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">4</span> <span class="chapter-title">Getting Started</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">5</span> <span class="chapter-title">Interpreting Images</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B3_Image_Series.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">6</span> <span class="chapter-title">Image Series</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">7</span> <span class="chapter-title">Vectors and Tables</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-3" aria-expanded="false">
|
||||
<span class="menu-text">C. Case Studies</span></a>
|
||||
<a class="sidebar-item-toggle text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-3" aria-expanded="false" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C1_Lights.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">8</span> <span class="chapter-title">War at Night</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C2_Refineries.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">9</span> <span class="chapter-title">Refinery Identification</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C3_Blast.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">10</span> <span class="chapter-title">Blast Damage Assessment</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C4_Ships.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">11</span> <span class="chapter-title">Ship Detection</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C5_Object_Detection.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">12</span> <span class="chapter-title">Object Detection</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</nav>
|
||||
<div id="quarto-sidebar-glass" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass"></div>
|
||||
<!-- margin-sidebar -->
|
||||
<div id="quarto-margin-sidebar" class="sidebar margin-sidebar">
|
||||
<nav id="TOC" role="doc-toc" class="toc-active">
|
||||
<h2 id="toc-title">Table of contents</h2>
|
||||
|
||||
<ul>
|
||||
<li><a href="#active-and-passive-sensors" id="toc-active-and-passive-sensors" class="nav-link active" data-scroll-target="#active-and-passive-sensors"><span class="header-section-number">2.1</span> Active and Passive Sensors</a></li>
|
||||
<li><a href="#resolution" id="toc-resolution" class="nav-link" data-scroll-target="#resolution"><span class="header-section-number">2.2</span> Resolution</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#spatial-resolution" id="toc-spatial-resolution" class="nav-link" data-scroll-target="#spatial-resolution"><span class="header-section-number">2.2.1</span> Spatial Resolution</a></li>
|
||||
<li><a href="#spectral-resolution" id="toc-spectral-resolution" class="nav-link" data-scroll-target="#spectral-resolution"><span class="header-section-number">2.2.2</span> Spectral Resolution</a></li>
|
||||
<li><a href="#temporal-resolution" id="toc-temporal-resolution" class="nav-link" data-scroll-target="#temporal-resolution"><span class="header-section-number">2.2.3</span> Temporal Resolution</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#summary" id="toc-summary" class="nav-link" data-scroll-target="#summary"><span class="header-section-number">2.3</span> Summary</a></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/A2_Remote_Sensing.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title"><span class="chapter-number">2</span> <span class="chapter-title">Remote Sensing</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="quarto-title-meta">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
</header>
|
||||
|
||||
<p>Before learning how to load, process, and analyze satellite imagery in Google Earth Engine, it will be helpful to know a few basic principles of remote sensing. This section provides a brief overview of some important concepts and terminology that will be used throughout the course, including active and passive sensors; spatial, spectral and temporal resolution; and orbits.</p>
|
||||
<section id="active-and-passive-sensors" class="level2" data-number="2.1">
|
||||
<h2 data-number="2.1" class="anchored" data-anchor-id="active-and-passive-sensors"><span class="header-section-number">2.1</span> Active and Passive Sensors</h2>
|
||||
<p><a href="https://www.sciencedirect.com/topics/medicine-and-dentistry/remote-sensing">Remote sensing</a> is the science of obtaining information about an object or phenomenon without making physical contact with the object. Remote sensing can be done with various types of electromagnetic radiation such as visible, infrared or microwave. The electromagnetic radiation is either emitted or reflected from the object being sensed. The reflected radiation is then collected by a sensor and processed to obtain information about the object.</p>
|
||||
<p><img src="../images/diagram.png" class="img-fluid"></p>
|
||||
<p>While most satellite imagery is optical, meaning it captures sunlight reflected by the earth’s surface, Synthetic Aperture Radar (SAR) satellites such as Sentinel-1 work by emitting pulses of radio waves and measuring how much of the signal is reflected back. This is similar to the way a bat uses sonar to “see” in the dark: by emitting calls and listening to echoes.</p>
|
||||
</section>
|
||||
<section id="resolution" class="level2" data-number="2.2">
|
||||
<h2 data-number="2.2" class="anchored" data-anchor-id="resolution"><span class="header-section-number">2.2</span> Resolution</h2>
|
||||
<p>Resolution is one of the most important attributes of satellite imagery. There are three types of resolution: spatial, spectral, and temporal. Let’s look at each of these.</p>
|
||||
<section id="spatial-resolution" class="level3" data-number="2.2.1">
|
||||
<h3 data-number="2.2.1" class="anchored" data-anchor-id="spatial-resolution"><span class="header-section-number">2.2.1</span> Spatial Resolution</h3>
|
||||
<p>Spatial resolution governs how “sharp” an image looks. The Google Maps satellite basemap, for example, is really sharp. Most of the optical imagery that is freely available has relatively low spatial resolution (it looks more grainy than, for example, the Google satellite basemap),</p>
|
||||
<p><img src="../images/Landsat.png" class="img-fluid"> <img src="../images/Sentinel2.png" class="img-fluid"> <img src="../images/Maxar.png" class="img-fluid"></p>
|
||||
</section>
|
||||
<section id="spectral-resolution" class="level3" data-number="2.2.2">
|
||||
<h3 data-number="2.2.2" class="anchored" data-anchor-id="spectral-resolution"><span class="header-section-number">2.2.2</span> Spectral Resolution</h3>
|
||||
<p>What open access imagery lacks in spatial resolution it often makes up for with <em>spectral</em> resolution. Really sharp imagery from MAXAR, for example, mostly collects light in the visible light spectrum, which is what our eyes can see. But there are other parts of the electromagnetic spectrum that we can’t see, but which can be very useful for distinguishing between different materials. Many satellites that have a lower spatial resolution than MAXAR, such as Landsat and Sentinel-2, collect data in a wider range of the electromagnetic spectrum.</p>
|
||||
<p>Different materials reflect light differently. An apple absorbs shorter wavelengths (e.g. blue and green), and reflects longer wavelengths (red). Our eyes use that information– the color– to distinguish between different objects. Below is a plot of the spectral profiles of different materials:</p>
|
||||
<iframe title="Spectral Profiles of Different Materials" aria-label="Interactive line chart" id="datawrapper-chart-b1kcX" src="https://datawrapper.dwcdn.net/b1kcX/3/" scrolling="no" frameborder="0" style="width: 0; min-width: 100% !important; border: none;" height="400">
|
||||
</iframe>
|
||||
<script type="text/javascript">!function(){"use strict";window.addEventListener("message",(function(e){if(void 0!==e.data["datawrapper-height"]){var t=document.querySelectorAll("iframe");for(var a in e.data["datawrapper-height"])for(var r=0;r<t.length;r++){if(t[r].contentWindow===e.source)t[r].style.height=e.data["datawrapper-height"][a]+"px"}}}))}();
|
||||
</script>
|
||||
<p>The visible portion of the spectrum is highlighted on the left, ranging from 400 nanometers (violet) to 700nm (red). Our eyes (and satellite imagery in the visible light spectrum) can only see this portion of the light spectrum; we can’t see UV or infrared wavelengths, for example, though the extent to which different materials reflect or absorb these wavelengths is just as useful for distinguishing between them. The European Space Agency’s Sentinel-2 satellite collects spectral information well beyond the visible light spectrum, enabling this sort of analysis. It chops the electromagnetic spectrum up into “bands”, and measures how strongly wavelengths in each of those bands is reflected:</p>
|
||||
<p><img src="images/S2_bands.png" class="img-fluid"></p>
|
||||
<p>To illustrate why this is important, consider Astroturf (fake plastic grass). Astroturf and real grass will both look green to us, especially from a satellite image. But living plants strongly reflect radiation from the sun in a part of the light spectrum that we can’t see (near-infrared). There’s a spectral index called the Normalized Difference Vegetation Index (NDVI) which exploits this fact to isolate vegetation in multispectral satellite imagery. So if we look at <a href="https://en.wikipedia.org/wiki/Gillette_Stadium">Gillette Stadium</a> near Boston, we can tell that the three training fields south of the stadium are real grass (they generate high NDVI values, showing up red), while the pitch in the stadium itself is astroturf (generating low NDVI values, showing up blue).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="images/NDVI.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">VHR image of Gillette Stadium with Sentinel-2 derived NDVI overlay</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>In other words, even though these fields are all green and indistinguishable to the human eye, their <em>spectral profiles</em> beyond the visible light spectrum differ, and we can use this information to distinguish between them.</p>
|
||||
<p>Astroturf is a trivial example. But suppose we were interested in identifying makeshift oil refineries in Northern Syria that constitute a key source of rents for whichever group controls them. As demonstrated in the <a href="./C2_Refineries.html">‘Refinery Identification’</a> case study, we can train an algorithm to identify the spectral signatures of oil, and use that to automatically detect them in satellite imagery.</p>
|
||||
</section>
|
||||
<section id="temporal-resolution" class="level3" data-number="2.2.3">
|
||||
<h3 data-number="2.2.3" class="anchored" data-anchor-id="temporal-resolution"><span class="header-section-number">2.2.3</span> Temporal Resolution</h3>
|
||||
<p>Finally, the frequency with which we can access new imagery is an important consideration. This is called the <strong>temporal resolution</strong>.</p>
|
||||
<p>The Google Maps basemap is very high resolution, available globally, and is freely available. But it has no <em>temporal</em> dimension: it’s a snapshot from one particular point in time. If the thing we’re interested in involves <em>changes</em> over time, this basemap will be of limited use.</p>
|
||||
<p>The <strong>“revisit rate”</strong> is the amount of time it takes for the satellite to pass over the same location twice. For example, the Sentinel-2 constellation’s two satellites can achieve a revisit rate of 5 days, as shown in this cool video from the European Space Agency:</p>
|
||||
<div class="quarto-video"><video id="video_shortcode_videojs_video1" class="video-js vjs-default-skin vjs-fluid" controls="" preload="auto" data-setup="{}" title=""><source src="https://dlmultimedia.esa.int/download/public/videos/2016/08/004/1608_004_AR_EN.mp4"></video></div>
|
||||
<p>Some satellite constellations are able to achieve much higher revisit rates. Sentinel-2 has a revisit rate of 5 days, but SkySat is capable of imaging the same point on earth around 12 times per day! How is that possible? Well, as the video above demonstrated, the Sentinel-2 constellation is composed of two satellites that share the same orbit, 180 degrees apart. In contrast, the SkySat constellation comprises 21 satellites, each with its own orbital path:</p>
|
||||
<div class="quarto-video"><video id="video_shortcode_videojs_video2" class="video-js vjs-default-skin vjs-fluid" controls="" preload="auto" data-setup="{}" title=""><source src="https://assets.planet.com/products/hi-res/Planet_Block_3_HD_1080p.mp4"></video></div>
|
||||
<p>This allows SkySat to achieve a revisit rate of 2-3 <em>hours</em>. The catch, however, is that you need to pay for it (and it <a href="https://apollomapping.com/blog/an-update-on-skysat-tasking-pricing-and-video-capabilities">ain’t cheap</a>). Below is a comparison of revisit rates for various other optical satellites:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="images/revisit_chart.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">A chart of revisit times for different satellites from <a href="https://link.springer.com/article/10.1007/s10712-021-09637-5">Sutlieff et. al.(2021)</a></figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="summary" class="level2" data-number="2.3">
|
||||
<h2 data-number="2.3" class="anchored" data-anchor-id="summary"><span class="header-section-number">2.3</span> Summary</h2>
|
||||
<p>You should hopefully have a better understanding of what satellite imagery is, and how it can be used to answer questions about the world. In the <a href="A3_Data_acquisition.qmd">next section</a>, we’ll look at the various types of satellite imagery stored in the Google Earth Engine catalog.</p>
|
||||
|
||||
|
||||
</section>
|
||||
|
||||
</main> <!-- /main -->
|
||||
<script id="quarto-html-after-body" type="application/javascript">
|
||||
window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
const toggleBodyColorMode = (bsSheetEl) => {
|
||||
const mode = bsSheetEl.getAttribute("data-mode");
|
||||
const bodyEl = window.document.querySelector("body");
|
||||
if (mode === "dark") {
|
||||
bodyEl.classList.add("quarto-dark");
|
||||
bodyEl.classList.remove("quarto-light");
|
||||
} else {
|
||||
bodyEl.classList.add("quarto-light");
|
||||
bodyEl.classList.remove("quarto-dark");
|
||||
}
|
||||
}
|
||||
const toggleBodyColorPrimary = () => {
|
||||
const bsSheetEl = window.document.querySelector("link#quarto-bootstrap");
|
||||
if (bsSheetEl) {
|
||||
toggleBodyColorMode(bsSheetEl);
|
||||
}
|
||||
}
|
||||
toggleBodyColorPrimary();
|
||||
const disableStylesheet = (stylesheets) => {
|
||||
for (let i=0; i < stylesheets.length; i++) {
|
||||
const stylesheet = stylesheets[i];
|
||||
stylesheet.rel = 'prefetch';
|
||||
}
|
||||
}
|
||||
const enableStylesheet = (stylesheets) => {
|
||||
for (let i=0; i < stylesheets.length; i++) {
|
||||
const stylesheet = stylesheets[i];
|
||||
stylesheet.rel = 'stylesheet';
|
||||
}
|
||||
}
|
||||
const manageTransitions = (selector, allowTransitions) => {
|
||||
const els = window.document.querySelectorAll(selector);
|
||||
for (let i=0; i < els.length; i++) {
|
||||
const el = els[i];
|
||||
if (allowTransitions) {
|
||||
el.classList.remove('notransition');
|
||||
} else {
|
||||
el.classList.add('notransition');
|
||||
}
|
||||
}
|
||||
}
|
||||
const toggleColorMode = (alternate) => {
|
||||
// Switch the stylesheets
|
||||
const alternateStylesheets = window.document.querySelectorAll('link.quarto-color-scheme.quarto-color-alternate');
|
||||
manageTransitions('#quarto-margin-sidebar .nav-link', false);
|
||||
if (alternate) {
|
||||
enableStylesheet(alternateStylesheets);
|
||||
for (const sheetNode of alternateStylesheets) {
|
||||
if (sheetNode.id === "quarto-bootstrap") {
|
||||
toggleBodyColorMode(sheetNode);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
disableStylesheet(alternateStylesheets);
|
||||
toggleBodyColorPrimary();
|
||||
}
|
||||
manageTransitions('#quarto-margin-sidebar .nav-link', true);
|
||||
// Switch the toggles
|
||||
const toggles = window.document.querySelectorAll('.quarto-color-scheme-toggle');
|
||||
for (let i=0; i < toggles.length; i++) {
|
||||
const toggle = toggles[i];
|
||||
if (toggle) {
|
||||
if (alternate) {
|
||||
toggle.classList.add("alternate");
|
||||
} else {
|
||||
toggle.classList.remove("alternate");
|
||||
}
|
||||
}
|
||||
}
|
||||
// Hack to workaround the fact that safari doesn't
|
||||
// properly recolor the scrollbar when toggling (#1455)
|
||||
if (navigator.userAgent.indexOf('Safari') > 0 && navigator.userAgent.indexOf('Chrome') == -1) {
|
||||
manageTransitions("body", false);
|
||||
window.scrollTo(0, 1);
|
||||
setTimeout(() => {
|
||||
window.scrollTo(0, 0);
|
||||
manageTransitions("body", true);
|
||||
}, 40);
|
||||
}
|
||||
}
|
||||
const isFileUrl = () => {
|
||||
return window.location.protocol === 'file:';
|
||||
}
|
||||
const hasAlternateSentinel = () => {
|
||||
let styleSentinel = getColorSchemeSentinel();
|
||||
if (styleSentinel !== null) {
|
||||
return styleSentinel === "alternate";
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
const setStyleSentinel = (alternate) => {
|
||||
const value = alternate ? "alternate" : "default";
|
||||
if (!isFileUrl()) {
|
||||
window.localStorage.setItem("quarto-color-scheme", value);
|
||||
} else {
|
||||
localAlternateSentinel = value;
|
||||
}
|
||||
}
|
||||
const getColorSchemeSentinel = () => {
|
||||
if (!isFileUrl()) {
|
||||
const storageValue = window.localStorage.getItem("quarto-color-scheme");
|
||||
return storageValue != null ? storageValue : localAlternateSentinel;
|
||||
} else {
|
||||
return localAlternateSentinel;
|
||||
}
|
||||
}
|
||||
let localAlternateSentinel = 'alternate';
|
||||
// Dark / light mode switch
|
||||
window.quartoToggleColorScheme = () => {
|
||||
// Read the current dark / light value
|
||||
let toAlternate = !hasAlternateSentinel();
|
||||
toggleColorMode(toAlternate);
|
||||
setStyleSentinel(toAlternate);
|
||||
};
|
||||
// Ensure there is a toggle, if there isn't float one in the top right
|
||||
if (window.document.querySelector('.quarto-color-scheme-toggle') === null) {
|
||||
const a = window.document.createElement('a');
|
||||
a.classList.add('top-right');
|
||||
a.classList.add('quarto-color-scheme-toggle');
|
||||
a.href = "";
|
||||
a.onclick = function() { try { window.quartoToggleColorScheme(); } catch {} return false; };
|
||||
const i = window.document.createElement("i");
|
||||
i.classList.add('bi');
|
||||
a.appendChild(i);
|
||||
window.document.body.appendChild(a);
|
||||
}
|
||||
// Switch to dark mode if need be
|
||||
if (hasAlternateSentinel()) {
|
||||
toggleColorMode(true);
|
||||
} else {
|
||||
toggleColorMode(false);
|
||||
}
|
||||
const icon = "";
|
||||
const anchorJS = new window.AnchorJS();
|
||||
anchorJS.options = {
|
||||
placement: 'right',
|
||||
icon: icon
|
||||
};
|
||||
anchorJS.add('.anchored');
|
||||
const isCodeAnnotation = (el) => {
|
||||
for (const clz of el.classList) {
|
||||
if (clz.startsWith('code-annotation-')) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
const clipboard = new window.ClipboardJS('.code-copy-button', {
|
||||
text: function(trigger) {
|
||||
const codeEl = trigger.previousElementSibling.cloneNode(true);
|
||||
for (const childEl of codeEl.children) {
|
||||
if (isCodeAnnotation(childEl)) {
|
||||
childEl.remove();
|
||||
}
|
||||
}
|
||||
return codeEl.innerText;
|
||||
}
|
||||
});
|
||||
clipboard.on('success', function(e) {
|
||||
// button target
|
||||
const button = e.trigger;
|
||||
// don't keep focus
|
||||
button.blur();
|
||||
// flash "checked"
|
||||
button.classList.add('code-copy-button-checked');
|
||||
var currentTitle = button.getAttribute("title");
|
||||
button.setAttribute("title", "Copied!");
|
||||
let tooltip;
|
||||
if (window.bootstrap) {
|
||||
button.setAttribute("data-bs-toggle", "tooltip");
|
||||
button.setAttribute("data-bs-placement", "left");
|
||||
button.setAttribute("data-bs-title", "Copied!");
|
||||
tooltip = new bootstrap.Tooltip(button,
|
||||
{ trigger: "manual",
|
||||
customClass: "code-copy-button-tooltip",
|
||||
offset: [0, -8]});
|
||||
tooltip.show();
|
||||
}
|
||||
setTimeout(function() {
|
||||
if (tooltip) {
|
||||
tooltip.hide();
|
||||
button.removeAttribute("data-bs-title");
|
||||
button.removeAttribute("data-bs-toggle");
|
||||
button.removeAttribute("data-bs-placement");
|
||||
}
|
||||
button.setAttribute("title", currentTitle);
|
||||
button.classList.remove('code-copy-button-checked');
|
||||
}, 1000);
|
||||
// clear code selection
|
||||
e.clearSelection();
|
||||
});
|
||||
function tippyHover(el, contentFn) {
|
||||
const config = {
|
||||
allowHTML: true,
|
||||
content: contentFn,
|
||||
maxWidth: 500,
|
||||
delay: 100,
|
||||
arrow: false,
|
||||
appendTo: function(el) {
|
||||
return el.parentElement;
|
||||
},
|
||||
interactive: true,
|
||||
interactiveBorder: 10,
|
||||
theme: 'quarto',
|
||||
placement: 'bottom-start'
|
||||
};
|
||||
window.tippy(el, config);
|
||||
}
|
||||
const noterefs = window.document.querySelectorAll('a[role="doc-noteref"]');
|
||||
for (var i=0; i<noterefs.length; i++) {
|
||||
const ref = noterefs[i];
|
||||
tippyHover(ref, function() {
|
||||
// use id or data attribute instead here
|
||||
let href = ref.getAttribute('data-footnote-href') || ref.getAttribute('href');
|
||||
try { href = new URL(href).hash; } catch {}
|
||||
const id = href.replace(/^#\/?/, "");
|
||||
const note = window.document.getElementById(id);
|
||||
return note.innerHTML;
|
||||
});
|
||||
}
|
||||
let selectedAnnoteEl;
|
||||
const selectorForAnnotation = ( cell, annotation) => {
|
||||
let cellAttr = 'data-code-cell="' + cell + '"';
|
||||
let lineAttr = 'data-code-annotation="' + annotation + '"';
|
||||
const selector = 'span[' + cellAttr + '][' + lineAttr + ']';
|
||||
return selector;
|
||||
}
|
||||
const selectCodeLines = (annoteEl) => {
|
||||
const doc = window.document;
|
||||
const targetCell = annoteEl.getAttribute("data-target-cell");
|
||||
const targetAnnotation = annoteEl.getAttribute("data-target-annotation");
|
||||
const annoteSpan = window.document.querySelector(selectorForAnnotation(targetCell, targetAnnotation));
|
||||
const lines = annoteSpan.getAttribute("data-code-lines").split(",");
|
||||
const lineIds = lines.map((line) => {
|
||||
return targetCell + "-" + line;
|
||||
})
|
||||
let top = null;
|
||||
let height = null;
|
||||
let parent = null;
|
||||
if (lineIds.length > 0) {
|
||||
//compute the position of the single el (top and bottom and make a div)
|
||||
const el = window.document.getElementById(lineIds[0]);
|
||||
top = el.offsetTop;
|
||||
height = el.offsetHeight;
|
||||
parent = el.parentElement.parentElement;
|
||||
if (lineIds.length > 1) {
|
||||
const lastEl = window.document.getElementById(lineIds[lineIds.length - 1]);
|
||||
const bottom = lastEl.offsetTop + lastEl.offsetHeight;
|
||||
height = bottom - top;
|
||||
}
|
||||
if (top !== null && height !== null && parent !== null) {
|
||||
// cook up a div (if necessary) and position it
|
||||
let div = window.document.getElementById("code-annotation-line-highlight");
|
||||
if (div === null) {
|
||||
div = window.document.createElement("div");
|
||||
div.setAttribute("id", "code-annotation-line-highlight");
|
||||
div.style.position = 'absolute';
|
||||
parent.appendChild(div);
|
||||
}
|
||||
div.style.top = top - 2 + "px";
|
||||
div.style.height = height + 4 + "px";
|
||||
let gutterDiv = window.document.getElementById("code-annotation-line-highlight-gutter");
|
||||
if (gutterDiv === null) {
|
||||
gutterDiv = window.document.createElement("div");
|
||||
gutterDiv.setAttribute("id", "code-annotation-line-highlight-gutter");
|
||||
gutterDiv.style.position = 'absolute';
|
||||
const codeCell = window.document.getElementById(targetCell);
|
||||
const gutter = codeCell.querySelector('.code-annotation-gutter');
|
||||
gutter.appendChild(gutterDiv);
|
||||
}
|
||||
gutterDiv.style.top = top - 2 + "px";
|
||||
gutterDiv.style.height = height + 4 + "px";
|
||||
}
|
||||
selectedAnnoteEl = annoteEl;
|
||||
}
|
||||
};
|
||||
const unselectCodeLines = () => {
|
||||
const elementsIds = ["code-annotation-line-highlight", "code-annotation-line-highlight-gutter"];
|
||||
elementsIds.forEach((elId) => {
|
||||
const div = window.document.getElementById(elId);
|
||||
if (div) {
|
||||
div.remove();
|
||||
}
|
||||
});
|
||||
selectedAnnoteEl = undefined;
|
||||
};
|
||||
// Attach click handler to the DT
|
||||
const annoteDls = window.document.querySelectorAll('dt[data-target-cell]');
|
||||
for (const annoteDlNode of annoteDls) {
|
||||
annoteDlNode.addEventListener('click', (event) => {
|
||||
const clickedEl = event.target;
|
||||
if (clickedEl !== selectedAnnoteEl) {
|
||||
unselectCodeLines();
|
||||
const activeEl = window.document.querySelector('dt[data-target-cell].code-annotation-active');
|
||||
if (activeEl) {
|
||||
activeEl.classList.remove('code-annotation-active');
|
||||
}
|
||||
selectCodeLines(clickedEl);
|
||||
clickedEl.classList.add('code-annotation-active');
|
||||
} else {
|
||||
// Unselect the line
|
||||
unselectCodeLines();
|
||||
clickedEl.classList.remove('code-annotation-active');
|
||||
}
|
||||
});
|
||||
}
|
||||
const findCites = (el) => {
|
||||
const parentEl = el.parentElement;
|
||||
if (parentEl) {
|
||||
const cites = parentEl.dataset.cites;
|
||||
if (cites) {
|
||||
return {
|
||||
el,
|
||||
cites: cites.split(' ')
|
||||
};
|
||||
} else {
|
||||
return findCites(el.parentElement)
|
||||
}
|
||||
} else {
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
var bibliorefs = window.document.querySelectorAll('a[role="doc-biblioref"]');
|
||||
for (var i=0; i<bibliorefs.length; i++) {
|
||||
const ref = bibliorefs[i];
|
||||
const citeInfo = findCites(ref);
|
||||
if (citeInfo) {
|
||||
tippyHover(citeInfo.el, function() {
|
||||
var popup = window.document.createElement('div');
|
||||
citeInfo.cites.forEach(function(cite) {
|
||||
var citeDiv = window.document.createElement('div');
|
||||
citeDiv.classList.add('hanging-indent');
|
||||
citeDiv.classList.add('csl-entry');
|
||||
var biblioDiv = window.document.getElementById('ref-' + cite);
|
||||
if (biblioDiv) {
|
||||
citeDiv.innerHTML = biblioDiv.innerHTML;
|
||||
}
|
||||
popup.appendChild(citeDiv);
|
||||
});
|
||||
return popup.innerHTML;
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./index.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">1</span> <span class="chapter-title">Overview</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./A3_Data_Acquisition.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-number">3</span> <span class="chapter-title">Data Acquisition</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
</div> <!-- /content -->
|
||||
<script>videojs(video_shortcode_videojs_video1);</script>
|
||||
<script>videojs(video_shortcode_videojs_video2);</script>
|
||||
|
||||
|
||||
|
||||
</body></html>
|
||||
1228
docs/A3_Data_Acquisition.html
Normal file
1228
docs/A3_Data_Acquisition.html
Normal file
File diff suppressed because it is too large
Load Diff
330
docs/A3_Data_acquisition.qmd
Normal file
330
docs/A3_Data_acquisition.qmd
Normal file
@@ -0,0 +1,330 @@
|
||||
---
|
||||
title: Data Acquisition
|
||||
---
|
||||
|
||||
One of the main advantages of GEE is that it hosts several Petabytes of satellite imagery and other spatial data sets, [all in one place](https://developers.google.com/earth-engine/datasets). Among these are many that could prove useful to those investigating illegal mining and logging, estimating conflict-induced damage, monitoring pollution from extractive industries, conducting maritime surveillance without relying on ship transponders, verifying the locations of artillery strikes, tracking missile defense systems and many other topics.
|
||||
|
||||
|
||||
This section highlights ten categories of geospatial data available natively in the GEE catalog, ranging from optical satellite imagery, to atmospheric data, to building footprints. Each sub-section provides an overview of the given data type, suggests potential applications, and lists the corresponding datasets in the GEE catalog. The datasets listed under each heading are **not** an exhaustive list-- there are over 500 in the whole catalog, and the ones listed in this section are simply the ones with the most immediate relevance to open source investigations. If a particular geospatial dataset you want to work with isn't hosted in the GEE catalog, you can upload your own data. We'll cover that in the next section.
|
||||
|
||||
|
||||
|
||||
|
||||
## Optical Imagery
|
||||
 tutorial.](../images/obj_det3.jpg)
|
||||
|
||||
|
||||
Optical satellite imagery is the bread and butter of many open source investigations. It would be tough to list off all of the possible use cases, so here's a handy flowchart:
|
||||
|
||||
|
||||
```{mermaid}
|
||||
%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#FFFFFF' ,'primaryBorderColor':'#000000' , 'lineColor':'#009933'}}}%%
|
||||
|
||||
|
||||
flowchart
|
||||
A(Does it happen outside?)
|
||||
A--> B(Yes)
|
||||
A--> C(No)
|
||||
D(Is it very small?)
|
||||
B-->D
|
||||
E(Yes)
|
||||
F(No)
|
||||
D-->F
|
||||
D-->E
|
||||
G(Use optical satellite imagery)
|
||||
H(Don't use optical satellite imagery)
|
||||
E-->H
|
||||
F-->G
|
||||
C-->H
|
||||
```
|
||||
|
||||
|
||||
This is, of course, a bit of an exaggeration. But if you're interested in a visible phenomenon that happens outdoors and that isn't very small, chances are an earth-observing satellite has taken a picture of it. What that picture can tell you naturally depends on what you're interested in learning. For a deeper dive into analyzing optical satellite imagery, see the subsection on [multispectral remote sensing.](A2_Remote_Sensing.qmd#multispectral-remote-sensing-remote_sensing).
|
||||
|
||||
|
||||
There are several different types of optical satellite imagery available in the GEE catalog. The main collections are the Landsat and Sentinel series of satellites, which are operated by NASA and the European Space Agency, respectively. Landsat satellites have been in orbit since 1972, and Sentinel satellites have been in orbit since 2015. Norway's International Climate and Forest Initiative (NICFI) has also contributed to the GEE catalog by providing a collection of optical imagery from Planet's PlanetScope satellites. These are higher resolution (4.7 meters per pixel) than Landsat (30m/px) and Sentinel-2 (10m/px), but are only available for the tropics. Even higher resolution imagery (60cm/px) is available from the GEE catalog from the National Agriculture Imagery Program, but it is only available for the United States. For more details, see the "Datasets" section below.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Geolocating pictures
|
||||
- Some of Bellingcat's [earliest work](https://www.bellingcat.com/resources/how-tos/2014/07/09/verification-and-geolocation-tricks-and-tips-with-google-earth/) involved figuring out where a picture was taken by cross-referencing it with optical satellite imagery.
|
||||
* General surveillance
|
||||
- [Monitoring](https://web.archive.org/web/20220415054905/https://fas.org/blogs/security/2021/11/a-closer-look-at-chinas-missile-silo-construction/) Chinese missile silo construction.
|
||||
- Amassing [evidence](https://www.nytimes.com/2022/04/04/world/europe/bucha-ukraine-bodies.html) of genocide in Bucha, Ukraine
|
||||
* Damage detection
|
||||
- [Ukraine](https://www.theguardian.com/world/2022/oct/27/before-and-after-satellite-imagery-will-track-ukraine-cultural-damage-un-says)
|
||||
- [Mali](https://reliefweb.int/report/mali/satellite-imagery-conflict-affected-areas-how-technology-can-support-wfp-emergency)
|
||||
- [Around the World](https://www.pnas.org/doi/pdf/10.1073/pnas.2025400118)
|
||||
* Verifying the locations of artillery/missile/drone strikes
|
||||
- The [2019 attack](https://www.cnbc.com/2019/09/17/satellite-photos-show-extent-of-damage-to-saudi-aramco-plants.html) on Saudi Arabia's Abqaiq oil processing facility.
|
||||
* Monitoring illegal mining/logging
|
||||
- Global Witness [investigation](https://www.globalwitness.org/en/campaigns/natural-resource-governance/myanmars-poisoned-mountains/) into illegal mining by militias in Myanmar.
|
||||
- Tracking [illegal logging](https://www.theguardian.com/environment/2016/mar/02/new-satellite-mapping-a-game-changer-against-illegal-logging) across the world.
|
||||
|
||||
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [Landsat 1-5](https://developers.google.com/earth-engine/datasets/catalog/landsat-mss) | 1972–1999 | 30m | Global |
|
||||
| [Landsat 7](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LE07_C02_T1_L2) | 1999–2021 | 30m | Global |
|
||||
| [Landsat 8](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC08_C02_T1_L2) | 2013–Present | 30m | Global |
|
||||
| [Landsat 9](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC09_C02_T1_L2) | 2021–Present | 30m | Global |
|
||||
| [Sentinel-2](https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR_HARMONIZED) | 2015–Present | 10m | Global |
|
||||
| [NICFI](https://developers.google.com/earth-engine/datasets/tags/nicfi) | 2015-Present | 4.7m | Tropics |
|
||||
| [NAIP](https://developers.google.com/earth-engine/datasets/catalog/USDA_NAIP_DOQQ) | 2002-2021 | 0.6m | USA |
|
||||
|
||||
|
||||
|
||||
|
||||
## Radar Imagery
|
||||
|
||||
|
||||
|
||||
Synthetic Aperture Radar imagery (SAR) is a type of remote sensing that uses radio waves to detect objects on the ground. SAR imagery is useful for detecting objects that are small, or that are obscured by clouds or other weather phenomena. SAR imagery is also useful for detecting objects that are moving, such as ships or cars.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Change/Damage detection
|
||||
* Tracking military radar systems
|
||||
* Maritime surveillance
|
||||
* Monitoring illegal mining/logging
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [Sentinel 1](https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S1_GRD) | 2014-Present | 10m | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
## Nighttime Lights
|
||||
|
||||
|
||||
|
||||
Satellite images of the Earth at night are a useful proxy for human activity. The brightness of a given area at night is a function of the number of people living there and the nature of their activities. The effects of conflict, natural disasters, and economic development can all be inferred from changes in nighttime lights.
|
||||
|
||||
|
||||
The timelapse above reveals a number of interesting things: The capture of Mosul by ISIS in 2014 and the destruction of its infrastructure during the fighting (shown as the city darkening), as well as the liberation of the city by the Iraqi military in 2017 are all visible in nighttime lights. The code to create this gif, as well as a more in-depth tutorial on the uses of nighttime lights, can be found in the ["War at Night"](C1_Lights.qmd) case study.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Damage detection
|
||||
* Identifying gas flaring/oil production
|
||||
* Identifying urban areas/military bases illuminated at night
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [DMSP-OLS](https://developers.google.com/earth-engine/datasets/catalog/NOAA_DMSP-OLS_NIGHTTIME_LIGHTS) | 1992-2014 | 927m | Global |
|
||||
| [VIIRS](https://developers.google.com/earth-engine/datasets/catalog/NOAA_VIIRS_DNB_MONTHLY_V1_VCMSLCFG) | 2014-Present | 463m | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Climate and Atmospheric Data
|
||||
|
||||
|
||||
|
||||
|
||||
{width=100%}
|
||||
|
||||
|
||||
Climate and atmospheric data can be used to track the effects of conflict on the environment. The European Space Agency's Sentinel-5p satellites measure the concentration of a number of atmospheric gasses, including nitrogen dioxide, methane and ozone. Measurements are available on a daily basis at a fairly high resolution (1km), allowing for the detection of localized sources of pollution such as oil refineries or power plants. For example, see this [Bellingcat article](https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/) in which Wim Zwijnenburg and I trace pollution to specific facilities operated by multinational oil companies in Iraq.
|
||||
|
||||
|
||||
The Copernicus Atmosphere Monitoring Service (CAMS) provides similar data at a lower spatial resolution (45km), but measurements are available on an hourly basis. The timelapse above utilizes CAMS data to show a sulfur dioxide plume resulting from an ISIS attack on the Al-Mishraq Sulphur Plant in Iraq. The plant was used to produce sulphuric acid, for use in fertilizers and pesticides. The attack destroyed the plant, causing a fire which burned for a month and released [21 kilotons](https://earthobservatory.nasa.gov/images/88994/sulfur-dioxide-spreads-over-iraq) of sulfur dioxide into the atmosphere per day; the largest human-made release of sulfur dioxide in history.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Monitoring of airborne pollution
|
||||
* Tracing pollution back to specific facilities and companies
|
||||
* Visualizing the effects of one-off environmental catastrophes
|
||||
- Nordstream 1 leak
|
||||
- ISIS setting Mishraq sulphur plant on fire
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [CAMS NRT](https://developers.google.com/earth-engine/datasets/catalog/ECMWF_CAMS_NRT) | 2016-Present | 44528m | Global |
|
||||
| [Sentinel-5p](https://developers.google.com/earth-engine/datasets/catalog/sentinel-5p) | 2018-Present | 1113m | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
## Mineral Deposits
|
||||
|
||||
|
||||
|
||||
Mining activities often play an important role in conflict. According to an influential [study](https://www.aeaweb.org/articles?id=10.1257/aer.20150774), "the historical rise in mineral prices might explain up to one-fourth of the average level of violence across African countries" between 1997 and 2010. Data on the location of mineral deposits can be used to identify areas where mining activities are likely to be taking place, and several such datasets are available in Google Earth Engine.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Monitoring mining activity
|
||||
* Identifying areas where mining activities are likely to be taking place
|
||||
* Mapping the distribution of resources in rebel held areas in conflicts fueled by resource extraction
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [iSDA](https://developers.google.com/earth-engine/datasets/tags/isda) | 2001-2017 | 30m | Africa |
|
||||
|
||||
|
||||
|
||||
|
||||
## Fires
|
||||
|
||||
|
||||
|
||||
Earth-observing satellites can detect "thermal anomalies" (fires) from space. NASA's Fire Information for Resource Management System (FIRMS) provides daily data on active fires in near real time, going back to the year 2000. Carlos Gonzales wrote a comprehensive [Bellingcat article](https://www.bellingcat.com/resources/2022/10/04/scorched-earth-using-nasa-fire-data-to-monitor-war-zones/) on the use of FIRMS to monitor war zones from Ukraine to Ethiopia. The map above shows that FIRMS detected fires over Eastern Ukraine trace the frontline of the war.
|
||||
|
||||
|
||||
FIRMS data are derived from the MODIS satellite, but only show the central location and intensity of a detected fire. Another MODIS product (linked in the table below) generates a monthly map of burned areas, which can be used to assess the spatial extent of fires.
|
||||
|
||||
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
* Identification of possible artillery strikes/fighting in places like Ukraine
|
||||
* Environmental warfare and "scorched earth" policies
|
||||
* Large scale arson
|
||||
- e.g. [Refugee camps burned down in Myanmar](https://citizenevidence.org/2021/02/26/using-viirs-fire-data-for-human-rights-research/)
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [FIRMS](https://developers.google.com/earth-engine/datasets/catalog/FIRMS) |2000-Present | 1000m | Global |
|
||||
| [MODIS Burned Area](https://developers.google.com/earth-engine/datasets/catalog/CIESIN_GPWv411_GPW_Population_Count) | 2000-Present | 500m | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
## Population Density Estimates
|
||||
|
||||
|
||||
|
||||
Sometimes, we may want to get an estimate of the population in a specific area to ballpark how many people might be affected by a natural disaster, a counteroffensive or a missile strike. You can't really Google "what is the population in this rectangle I've drawn in Northeastern Syria?" and get a good answer. Luckily, there are several spatial population datasets hosted in GEE that let you do just that. Some, such as WorldPop, provide estimated breakdowns by age and sex as well. However, it is extremely important to bear in mind that these are **estimates**, and will **not** take into account things like conflict-induced displacement. For example, Oak Ridge National Laboratory's LandScan program has released high-resolution population data for Ukraine, but this pertains to the pre-war population distribution. The war has radically changed this distribution, so these estimates no longer reflect where people *are*. Still, this dataset could be used to roughly estimate displacement or the number of people who will need new housing.
|
||||
|
||||
|
||||
### Applications: {.unnumbered}
|
||||
|
||||
|
||||
* Rough estimates of civilians at risk from conflict or disaster, provided at a high spatial resolution
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Sensor | Timeframe | Resolution | Coverage |
|
||||
| ----------- | ------------ | ---------- | -------- |
|
||||
| [Worldpop](https://developers.google.com/earth-engine/datasets/tags/worldpop) |2000-2021 | 92m | Global |
|
||||
| [GPW](https://developers.google.com/earth-engine/datasets/catalog/CIESIN_GPWv411_GPW_Population_Count) | 2000-2021 | 927m | Global |
|
||||
| [LandScan](https://developers.google.com/earth-engine/datasets/catalog/DOE_ORNL_LandScan_HD_Ukraine_202201) | 2013–Present | 100m | Ukraine |
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Building Footprints
|
||||
|
||||
|
||||
|
||||
A building footprint dataset contains the two dimensional outlines of buildings in a given area. Currently, GEE hosts one building footprint dataset which covers all of Africa. In 2022, Microsoft released a free [global building footprint dataset](https://www.microsoft.com/en-us/maps/building-footprints), though to use it in Earth Engine you'll have to download it from their [GitHub page](https://github.com/Microsoft/USBuildingFootprints) and upload it manually to GEE. The same goes for OpenStreetMap (OSM), a public database of building footprints, roads, and other features that also contains useful annotations for many buildings indicating their use. [Benjamin Strick](https://www.youtube.com/watch?v=bJkV3l5Haq0) has a great youtube video on conducting investigations using OSM data.
|
||||
|
||||
|
||||
### Applications: {.unnumbered}
|
||||
|
||||
|
||||
* Joining damage estimate data with the number of buildings in an area
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Dataset | Timeframe | Coverage |
|
||||
| ----------- | ------------ | -------- |
|
||||
| [Open Buildings](https://developers.google.com/earth-engine/datasets/catalog/GOOGLE_Research_open-buildings_v2_polygons) |2022 | Africa |
|
||||
|
||||
|
||||
|
||||
|
||||
## Administrative Boundaries
|
||||
|
||||
|
||||
|
||||
Spatial analysis often has to aggregate information over a defined area; we may want to assess the total burned area by province in Ukraine, or count the number of Saudi airstrikes by district in Yemen. For that, we need data on these administrative boundaries. GEE hosts several such datasets at the country, province, and district (or equivalent) level.
|
||||
|
||||
|
||||
### Applications {.unnumbered}
|
||||
|
||||
|
||||
* Quick spatial calculations for different provinces/districts in a country
|
||||
- e.g. counts of conflict events by district over time
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Dataset | Timeframe | Coverage |
|
||||
| ----------- | ------------ | -------- |
|
||||
| [FAO GAUL](https://developers.google.com/earth-engine/datasets/tags/gaul) |2015 | Global |
|
||||
|
||||
|
||||
|
||||
|
||||
## Global Power Plant Database
|
||||
|
||||
|
||||
|
||||
The Global Power Plant Database is a comprehensive, open source database of power plants around the world. It centralizes power plant data to make it easier to navigate, compare and draw insights. Each power plant is geolocated and entries contain information on plant capacity, generation, ownership, and fuel type. As of June 2018, the database includes around 28,500 power plants from 164 countries. The database is curated by the [World Resources Institute (WRI)](https://datasets.wri.org/dataset/globalpowerplantdatabase).
|
||||
|
||||
|
||||
### Applications: {.unnumbered}
|
||||
|
||||
|
||||
* Analyzing the impact of conflict on critical infrastructure.
|
||||
- e.g. fighting in Ukraine taking place around nuclear power facilities.
|
||||
* Could be combined with the atmospheric measurements of different pollutants and the population estimates data to assess the impact of various forms of energy generation on air quality and public health.
|
||||
|
||||
|
||||
### Datasets {.unnumbered}
|
||||
|
||||
|
||||
| Dataset | Timeframe | Coverage |
|
||||
| ----------- | ------------ | -------- |
|
||||
| [GPPD](https://developers.google.com/earth-engine/datasets/catalog/WRI_GPPD_power_plants) |2018 | Global |
|
||||
2163
docs/B1_Getting_Started.html
Normal file
2163
docs/B1_Getting_Started.html
Normal file
File diff suppressed because it is too large
Load Diff
1735
docs/B2_Interpreting_Images.html
Normal file
1735
docs/B2_Interpreting_Images.html
Normal file
File diff suppressed because it is too large
Load Diff
3869
docs/B3_Image_Series.html
Normal file
3869
docs/B3_Image_Series.html
Normal file
File diff suppressed because it is too large
Load Diff
2599
docs/B4_Vectors_Tables.html
Normal file
2599
docs/B4_Vectors_Tables.html
Normal file
File diff suppressed because it is too large
Load Diff
933
docs/C1_Lights.html
Normal file
933
docs/C1_Lights.html
Normal file
@@ -0,0 +1,933 @@
|
||||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta name="generator" content="quarto-1.3.326">
|
||||
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - War at Night</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
div.columns{display: flex; gap: min(4vw, 1.5em);}
|
||||
div.column{flex: auto; overflow-x: auto;}
|
||||
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
||||
ul.task-list{list-style: none;}
|
||||
ul.task-list li input[type="checkbox"] {
|
||||
width: 0.8em;
|
||||
margin: 0 0.8em 0.2em -1em; /* quarto-specific, see https://github.com/quarto-dev/quarto-cli/issues/4556 */
|
||||
vertical-align: middle;
|
||||
}
|
||||
/* CSS for syntax highlighting */
|
||||
pre > code.sourceCode { white-space: pre; position: relative; }
|
||||
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
|
||||
pre > code.sourceCode > span:empty { height: 1.2em; }
|
||||
.sourceCode { overflow: visible; }
|
||||
code.sourceCode > span { color: inherit; text-decoration: inherit; }
|
||||
div.sourceCode { margin: 1em 0; }
|
||||
pre.sourceCode { margin: 0; }
|
||||
@media screen {
|
||||
div.sourceCode { overflow: auto; }
|
||||
}
|
||||
@media print {
|
||||
pre > code.sourceCode { white-space: pre-wrap; }
|
||||
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
|
||||
}
|
||||
pre.numberSource code
|
||||
{ counter-reset: source-line 0; }
|
||||
pre.numberSource code > span
|
||||
{ position: relative; left: -4em; counter-increment: source-line; }
|
||||
pre.numberSource code > span > a:first-child::before
|
||||
{ content: counter(source-line);
|
||||
position: relative; left: -1em; text-align: right; vertical-align: baseline;
|
||||
border: none; display: inline-block;
|
||||
-webkit-touch-callout: none; -webkit-user-select: none;
|
||||
-khtml-user-select: none; -moz-user-select: none;
|
||||
-ms-user-select: none; user-select: none;
|
||||
padding: 0 4px; width: 4em;
|
||||
}
|
||||
pre.numberSource { margin-left: 3em; padding-left: 4px; }
|
||||
div.sourceCode
|
||||
{ }
|
||||
@media screen {
|
||||
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
|
||||
}
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./C2_Refineries.html" rel="next">
|
||||
<link href="./B4_Vectors_Tables.html" rel="prev">
|
||||
<link href="./../favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
"collapse-after": 3,
|
||||
"panel-placement": "start",
|
||||
"type": "textbox",
|
||||
"limit": 20,
|
||||
"language": {
|
||||
"search-no-results-text": "No results",
|
||||
"search-matching-documents-text": "matching documents",
|
||||
"search-copy-link-title": "Copy link to search",
|
||||
"search-hide-matches-text": "Hide additional matches",
|
||||
"search-more-match-text": "more match in this document",
|
||||
"search-more-matches-text": "more matches in this document",
|
||||
"search-clear-button-title": "Clear",
|
||||
"search-detached-cancel-button-title": "Cancel",
|
||||
"search-submit-button-title": "Submit"
|
||||
}
|
||||
}</script>
|
||||
<script async="" src="https://www.googletagmanager.com/gtag/js?id=G-RK9ZLZQ6GL"></script>
|
||||
|
||||
<script type="text/javascript">
|
||||
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</script>
|
||||
|
||||
|
||||
</head>
|
||||
|
||||
<body class="nav-sidebar floating">
|
||||
|
||||
|
||||
<div id="quarto-search-results"></div>
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav">
|
||||
<div class="container-fluid d-flex">
|
||||
<button type="button" class="quarto-btn-toggle btn" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<i class="bi bi-layout-text-sidebar-reverse"></i>
|
||||
</button>
|
||||
<nav class="quarto-page-breadcrumbs" aria-label="breadcrumb"><ol class="breadcrumb"><li class="breadcrumb-item"><a href="./C1_Lights.html">C. Case Studies</a></li><li class="breadcrumb-item"><a href="./C1_Lights.html"><span class="chapter-number">8</span> <span class="chapter-title">War at Night</span></a></li></ol></nav>
|
||||
<a class="flex-grow-1" role="button" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
</a>
|
||||
<button type="button" class="btn quarto-search-button" aria-label="Search" onclick="window.quartoOpenSearch();">
|
||||
<i class="bi bi-search"></i>
|
||||
</button>
|
||||
</div>
|
||||
</nav>
|
||||
</header>
|
||||
<!-- content -->
|
||||
<div id="quarto-content" class="quarto-container page-columns page-rows-contents page-layout-article">
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse collapse-horizontal sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="quarto-navigation-tool px-1" aria-label="Source Code"><i class="bi bi-github"></i></a>
|
||||
<div class="dropdown">
|
||||
<a href="" title="Download" id="quarto-navigation-tool-dropdown-0" class="quarto-navigation-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Download"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="quarto-navigation-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="dropdown">
|
||||
<a href="" title="Share" id="quarto-navigation-tool-dropdown-1" class="quarto-navigation-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Share"><i class="bi bi-share"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="quarto-navigation-tool-dropdown-1">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="https://twitter.com/intent/tweet?url=|url|">
|
||||
<i class="bi bi-bi-twitter pe-1"></i>
|
||||
Twitter
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="https://www.facebook.com/sharer/sharer.php?u=|url|">
|
||||
<i class="bi bi-bi-facebook pe-1"></i>
|
||||
Facebook
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<a href="" class="quarto-color-scheme-toggle quarto-navigation-tool px-1" onclick="window.quartoToggleColorScheme(); return false;" title="Toggle dark mode"><i class="bi"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-2 flex-shrink-0 align-items-center">
|
||||
<div class="sidebar-search">
|
||||
<div id="quarto-search" class="" title="Search"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="sidebar-menu-container">
|
||||
<ul class="list-unstyled mt-1">
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-1" aria-expanded="false">
|
||||
<span class="menu-text">A. Introduction</span></a>
|
||||
<a class="sidebar-item-toggle text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-1" aria-expanded="false" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Overview</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-2" aria-expanded="false">
|
||||
<span class="menu-text">B. Google Earth Engine</span></a>
|
||||
<a class="sidebar-item-toggle text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-2" aria-expanded="false" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-3" aria-expanded="true">
|
||||
<span class="menu-text">C. Case Studies</span></a>
|
||||
<a class="sidebar-item-toggle text-start" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-3" aria-expanded="true" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C1_Lights.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</nav>
|
||||
<div id="quarto-sidebar-glass" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass"></div>
|
||||
<!-- margin-sidebar -->
|
||||
<div id="quarto-margin-sidebar" class="sidebar margin-sidebar">
|
||||
<nav id="TOC" role="doc-toc" class="toc-active">
|
||||
<h2 id="toc-title">Table of contents</h2>
|
||||
|
||||
<ul>
|
||||
<li><a href="#pre-processing" id="toc-pre-processing" class="nav-link active" data-scroll-target="#pre-processing">Pre-Processing</a></li>
|
||||
<li><a href="#analysis" id="toc-analysis" class="nav-link" data-scroll-target="#analysis">Analysis</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#the-fall-of-mosul" id="toc-the-fall-of-mosul" class="nav-link" data-scroll-target="#the-fall-of-mosul">The Fall of Mosul</a></li>
|
||||
<li><a href="#the-qayyarah-fires" id="toc-the-qayyarah-fires" class="nav-link" data-scroll-target="#the-qayyarah-fires">The Qayyarah Fires</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/C1_Lights.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title"><span class="chapter-title">War at Night</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="quarto-title-meta">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
</header>
|
||||
|
||||
<p>Satellite images of Syria taken at night capture a subtle trace left by human civilization: lights. Apartment buildings, street lights, highways, power plants – all are illuminated at night and can be seen from space. Researchers often use these nighttime lights signatures to track development; as cities grow, villages receive power and infrastructure is built, areas emit more light. But this works both ways. As cities are demolished, villages burned and highways cutoff, they stop emitting lights.</p>
|
||||
<p>In this tutorial, we’ll use satellite images of Iraq taken at night to track the destruction caused by the fight against the Islamic State. We’ll use the VIIRS nighttime lights dataset, which is a collection of satellite images taken by the Visible Infrared Imaging Radiometer Suite (VIIRS) on the Suomi NPP satellite. VIIRS is a sensor that can detect light in the visible and infrared spectrum, and is capable of taking images at night. A link to the GEE code for this section can be found <a href="https://code.earthengine.google.com/2cf77d8cb9afd76b73100637fbffdf5d">here</a>.</p>
|
||||
<section id="pre-processing" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="pre-processing">Pre-Processing</h2>
|
||||
<p>First, let’s start by importing a few useful packages written by <a href="https://twitter.com/gena_d">Gennadii Donchyts</a>. We’ll use <code>utils</code> and <code>text</code> to annotate the date of each image on the timelapse. We’ll also define an Area of Interest (AOI), which is just a rectangle. You can do this manually by clicking the drawing tools in the top left. I’ve drawn an AOI over the area covering Mosul, Irbil and Kirkuk in Northern Iraq.</p>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> utils <span class="op">=</span> <span class="pp">require</span>(<span class="st">"users/gena/packages:utils"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> text <span class="op">=</span> <span class="pp">require</span>(<span class="st">"users/gena/packages:text"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="co">// define the Area of Interest (AOI)</span></span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> AOI <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">42.555362833405326</span><span class="op">,</span> <span class="fl">36.62010778397765</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> [<span class="fl">42.555362833405326</span><span class="op">,</span> <span class="fl">35.18296243288332</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> [<span class="fl">44.681217325592826</span><span class="op">,</span> <span class="fl">35.18296243288332</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> [<span class="fl">44.681217325592826</span><span class="op">,</span> <span class="fl">36.62010778397765</span>]]])</span>
|
||||
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="co">// start and end dates for our gif </span></span>
|
||||
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> startDate <span class="op">=</span> <span class="st">'2013-01-01'</span><span class="op">;</span></span>
|
||||
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> endDate <span class="op">=</span> <span class="st">'2018-01-01'</span><span class="op">;</span></span>
|
||||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a><span class="co">// a filename for when we export the gif</span></span>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> export_name<span class="op">=</span><span class="st">'qayyarah_viirs'</span></span>
|
||||
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a><span class="co">// A palette to visualize the VIIRS imagery. This one is similar to Matplotlib's "Magma" palette. </span></span>
|
||||
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> viirs_palette <span class="op">=</span> [</span>
|
||||
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a> <span class="st">"#000004"</span><span class="op">,</span></span>
|
||||
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a> <span class="st">"#320a5a"</span><span class="op">,</span></span>
|
||||
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a> <span class="st">"#781b6c"</span><span class="op">,</span></span>
|
||||
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a> <span class="st">"#bb3654"</span><span class="op">,</span></span>
|
||||
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a> <span class="st">"#ec6824"</span><span class="op">,</span></span>
|
||||
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a> <span class="st">"#fbb41a"</span><span class="op">,</span></span>
|
||||
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a> <span class="st">"#fcffa4"</span><span class="op">,</span></span>
|
||||
<span id="cb1-30"><a href="#cb1-30" aria-hidden="true" tabindex="-1"></a>]<span class="op">;</span></span>
|
||||
<span id="cb1-31"><a href="#cb1-31" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-32"><a href="#cb1-32" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-33"><a href="#cb1-33" aria-hidden="true" tabindex="-1"></a><span class="co">// Visualisation parameters for the VIIRS imagery, defining a minimum and maximum value, and referencing the palette we just created</span></span>
|
||||
<span id="cb1-34"><a href="#cb1-34" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> VIIRSvis <span class="op">=</span> { <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="fl">0.1</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="fl">1.6</span><span class="op">,</span> <span class="dt">palette</span><span class="op">:</span> viirs_palette }<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Next, we’ll load the VIIRS nighttime lights imagery. We want to select the <code>avg_rad</code> band of the image collection, and filter blank images. Sometimes, we get blank images over an area in VIIRS if our AOI is on the edge of the satellite’s imaging swath. We can filter these images, similarly to how we filter for cloudy images in Sentinel-2:</p>
|
||||
<div class="sourceCode" id="cb2"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> VIIRS<span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">"NOAA/VIIRS/DNB/MONTHLY_V1/VCMCFG"</span>) </span>
|
||||
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">select</span>(<span class="st">'avg_rad'</span>)</span>
|
||||
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the sum of the 'avg_rad' band within the AOI</span></span>
|
||||
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span>(image) { </span>
|
||||
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> blank<span class="op">=</span>image<span class="op">.</span><span class="fu">reduceRegions</span>({ <span class="co">// reduceRegions is a function that allows us to reduce the values of a band within a</span></span>
|
||||
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> AOI<span class="op">,</span> <span class="co">// geometry. In this case, we're reducing the values of the 'avg_rad' band within the AOI</span></span>
|
||||
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">reducer</span><span class="op">:</span> ee<span class="op">.</span><span class="at">Reducer</span><span class="op">.</span><span class="fu">sum</span>()<span class="op">,</span> <span class="co">// We're using the sum reducer, which will sum the values of the 'avg_rad' band</span></span>
|
||||
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> <span class="dv">10</span>}) <span class="co">// We're reducing the values of the 'avg_rad' band at a scale of 10m</span></span>
|
||||
<span id="cb2-9"><a href="#cb2-9" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">first</span>() <span class="co">// We only want the first element of the collection, which is the sum of the 'avg_rad' band within the AOI</span></span>
|
||||
<span id="cb2-10"><a href="#cb2-10" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">get</span>(<span class="st">'sum'</span>) <span class="co">// We want the value of the 'sum' property, which is the sum of the 'avg_rad' band within the AOI</span></span>
|
||||
<span id="cb2-11"><a href="#cb2-11" aria-hidden="true" tabindex="-1"></a> <span class="co">// For each image, define a property 'blank' that stores the sum of the 'avg_rad' band within the AOI. </span></span>
|
||||
<span id="cb2-12"><a href="#cb2-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// We're also going to take a base 10 log of the image-- this will help us visualize the data by dampening extreme values </span></span>
|
||||
<span id="cb2-13"><a href="#cb2-13" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> image<span class="op">.</span><span class="fu">set</span>(<span class="st">'blank'</span><span class="op">,</span> blank)<span class="op">.</span><span class="fu">log10</span>()<span class="op">.</span><span class="fu">unmask</span>(<span class="dv">0</span>)</span>
|
||||
<span id="cb2-14"><a href="#cb2-14" aria-hidden="true" tabindex="-1"></a> })</span>
|
||||
<span id="cb2-15"><a href="#cb2-15" aria-hidden="true" tabindex="-1"></a> <span class="co">// Now, we can filter images which are fully or partially blank over our AOI</span></span>
|
||||
<span id="cb2-16"><a href="#cb2-16" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">gt</span>(<span class="st">'blank'</span><span class="op">,</span> <span class="dv">10</span>))</span>
|
||||
<span id="cb2-17"><a href="#cb2-17" aria-hidden="true" tabindex="-1"></a> <span class="co">// Finally, we filter the collection to the specified date range</span></span>
|
||||
<span id="cb2-18"><a href="#cb2-18" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filterDate</span>(startDate<span class="op">,</span> endDate)</span>
|
||||
<span id="cb2-19"><a href="#cb2-19" aria-hidden="true" tabindex="-1"></a> </span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Let’s have a look at the first image in the collection to make sure everything’s looking right. We’ll set the basemap to satellite and center our AOI:</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setOptions</span>(<span class="st">'HYBRID'</span>)</span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">centerObject</span>(AOI)</span>
|
||||
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(VIIRS<span class="op">.</span><span class="fu">first</span>()<span class="op">,</span>VIIRSvis<span class="op">,</span><span class="st">'Nighttime Lights'</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="../images/iraq_check.png" class="img-fluid"></p>
|
||||
<p>If we decrease the opacity of the VIIRS layer, we can see the cities of Mosul, Erbil and Kirkuk shining brightly at night. We can also see a string of bright lights between Kirkuk and Erbil – these are methane flares from oil wells.</p>
|
||||
</section>
|
||||
<section id="analysis" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="analysis">Analysis</h2>
|
||||
<p>Having pre-processed the VIIRS imagery, we can now define a function <code>gif</code> that will take:</p>
|
||||
<ol type="1">
|
||||
<li>An image collection (<code>col</code>, in this case the nighttime lights imagery <code>VIIRS</code>)</li>
|
||||
<li>Visualization parameters (<code>col_vis</code>, in this case <code>VIIRSvis</code>)</li>
|
||||
<li>An Area of Interest <code>AOI</code></li>
|
||||
</ol>
|
||||
<p>The function will then return a timelapse.</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> gif <span class="op">=</span> <span class="kw">function</span> (col<span class="op">,</span> col_vis<span class="op">,</span> AOI) {</span>
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="co">// Define the date annotations to be printed in the top left of the gif in white</span></span>
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> annotations <span class="op">=</span> [</span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">textColor</span><span class="op">:</span> <span class="st">"white"</span><span class="op">,</span></span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">position</span><span class="op">:</span> <span class="st">"left"</span><span class="op">,</span></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">offset</span><span class="op">:</span> <span class="st">"1%"</span><span class="op">,</span></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">margin</span><span class="op">:</span> <span class="st">"1%"</span><span class="op">,</span></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">property</span><span class="op">:</span> <span class="st">"label"</span><span class="op">,</span></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Dynamically size the annotations according to the size of the AOI</span></span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> AOI<span class="op">.</span><span class="fu">area</span>(<span class="dv">100</span>)<span class="op">.</span><span class="fu">sqrt</span>()<span class="op">.</span><span class="fu">divide</span>(<span class="dv">200</span>)<span class="op">,</span></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> ]<span class="op">;</span></span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a> <span class="co">// Next, we want to map over the image collection,</span></span>
|
||||
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> rgbVis <span class="op">=</span> col<span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span> (image) {</span>
|
||||
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the date of the image and format it</span></span>
|
||||
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> start <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(image<span class="op">.</span><span class="fu">get</span>(<span class="st">"system:time_start"</span>))<span class="op">;</span></span>
|
||||
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> label <span class="op">=</span> start<span class="op">.</span><span class="fu">format</span>(<span class="st">"YYYY-MM-dd"</span>)<span class="op">;</span></span>
|
||||
<span id="cb4-23"><a href="#cb4-23" aria-hidden="true" tabindex="-1"></a> <span class="co">// And visualize the image using the visualization parameters defined earlier.</span></span>
|
||||
<span id="cb4-24"><a href="#cb4-24" aria-hidden="true" tabindex="-1"></a> <span class="co">// We also want to set a property called "label" that stores the formatted date </span></span>
|
||||
<span id="cb4-25"><a href="#cb4-25" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> image<span class="op">.</span><span class="fu">visualize</span>(col_vis)<span class="op">.</span><span class="fu">set</span>({ <span class="dt">label</span><span class="op">:</span> label })<span class="op">;</span></span>
|
||||
<span id="cb4-26"><a href="#cb4-26" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-27"><a href="#cb4-27" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-28"><a href="#cb4-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-29"><a href="#cb4-29" aria-hidden="true" tabindex="-1"></a> <span class="co">// Now we use the label property and the annotateImage function from @gena_d to annotate each image with the date. </span></span>
|
||||
<span id="cb4-30"><a href="#cb4-30" aria-hidden="true" tabindex="-1"></a> rgbVis <span class="op">=</span> rgbVis<span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span> (image) {</span>
|
||||
<span id="cb4-31"><a href="#cb4-31" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> text<span class="op">.</span><span class="fu">annotateImage</span>(image<span class="op">,</span> {}<span class="op">,</span> AOI<span class="op">,</span> annotations)<span class="op">;</span></span>
|
||||
<span id="cb4-32"><a href="#cb4-32" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-33"><a href="#cb4-33" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-34"><a href="#cb4-34" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-35"><a href="#cb4-35" aria-hidden="true" tabindex="-1"></a> <span class="co">// Define GIF visualization parameters.</span></span>
|
||||
<span id="cb4-36"><a href="#cb4-36" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> gifParams <span class="op">=</span> {</span>
|
||||
<span id="cb4-37"><a href="#cb4-37" aria-hidden="true" tabindex="-1"></a> <span class="dt">maxPixels</span><span class="op">:</span> <span class="dv">27017280</span><span class="op">,</span></span>
|
||||
<span id="cb4-38"><a href="#cb4-38" aria-hidden="true" tabindex="-1"></a> <span class="dt">region</span><span class="op">:</span> AOI<span class="op">,</span></span>
|
||||
<span id="cb4-39"><a href="#cb4-39" aria-hidden="true" tabindex="-1"></a> <span class="dt">crs</span><span class="op">:</span> <span class="st">"EPSG:3857"</span><span class="op">,</span></span>
|
||||
<span id="cb4-40"><a href="#cb4-40" aria-hidden="true" tabindex="-1"></a> <span class="dt">dimensions</span><span class="op">:</span> <span class="dv">640</span><span class="op">,</span></span>
|
||||
<span id="cb4-41"><a href="#cb4-41" aria-hidden="true" tabindex="-1"></a> <span class="dt">framesPerSecond</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span></span>
|
||||
<span id="cb4-42"><a href="#cb4-42" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
|
||||
<span id="cb4-43"><a href="#cb4-43" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-44"><a href="#cb4-44" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-45"><a href="#cb4-45" aria-hidden="true" tabindex="-1"></a> <span class="co">// Export the gif to Google Drive</span></span>
|
||||
<span id="cb4-46"><a href="#cb4-46" aria-hidden="true" tabindex="-1"></a> Export<span class="op">.</span><span class="at">video</span><span class="op">.</span><span class="fu">toDrive</span>({</span>
|
||||
<span id="cb4-47"><a href="#cb4-47" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> rgbVis<span class="op">,</span> <span class="co">// the image collection</span></span>
|
||||
<span id="cb4-48"><a href="#cb4-48" aria-hidden="true" tabindex="-1"></a> <span class="dt">description</span><span class="op">:</span> export_name<span class="op">,</span> <span class="co">// the name of the file</span></span>
|
||||
<span id="cb4-49"><a href="#cb4-49" aria-hidden="true" tabindex="-1"></a> <span class="dt">dimensions</span><span class="op">:</span> <span class="dv">1080</span><span class="op">,</span> <span class="co">// the dimensions of the gif</span></span>
|
||||
<span id="cb4-50"><a href="#cb4-50" aria-hidden="true" tabindex="-1"></a> <span class="dt">framesPerSecond</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span> <span class="co">// the number of frames per second</span></span>
|
||||
<span id="cb4-51"><a href="#cb4-51" aria-hidden="true" tabindex="-1"></a> <span class="dt">region</span><span class="op">:</span> AOI<span class="op">,</span> <span class="co">// the area of interest</span></span>
|
||||
<span id="cb4-52"><a href="#cb4-52" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-53"><a href="#cb4-53" aria-hidden="true" tabindex="-1"></a> <span class="co">// Print the GIF URL to the console.</span></span>
|
||||
<span id="cb4-54"><a href="#cb4-54" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(rgbVis<span class="op">.</span><span class="fu">getVideoThumbURL</span>(gifParams))<span class="op">;</span></span>
|
||||
<span id="cb4-55"><a href="#cb4-55" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-56"><a href="#cb4-56" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-57"><a href="#cb4-57" aria-hidden="true" tabindex="-1"></a> <span class="co">// Render the GIF animation in the console.</span></span>
|
||||
<span id="cb4-58"><a href="#cb4-58" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(ui<span class="op">.</span><span class="fu">Thumbnail</span>(rgbVis<span class="op">,</span> gifParams))<span class="op">;</span></span>
|
||||
<span id="cb4-59"><a href="#cb4-59" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Ok that was a pretty big chunk of code. But the good news is that we basically never have to touch it again, since we can just feed it different inputs. For example, if I want to generate a gif of night time lights over a different area, it’s as simple as dragging the AOI. If I want to look at a different time period, I can just edit the <code>startDate</code> and <code>endDate</code> variables. And if I want to visualize an entirely different type of satellite imagery – Sentinel-1, Sentinel-2, or anything else, all I have to do is change the image collection (<code>col</code>) and visualization parameters (<code>col_vis</code>) variables. Now, let’s look at some timelapses.</p>
|
||||
<section id="the-fall-of-mosul" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="the-fall-of-mosul">The Fall of Mosul</h3>
|
||||
<p>The function returns a timelapse of nighttime lights over Northern Iraq:</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="fu">gif</span>(VIIRS<span class="op">,</span> VIIRSvis<span class="op">,</span> AOI)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="../images/Figure_1.gif" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">I’ve done a bit of post-processing to this gif, adding more annotations and blending between frames to make it a bit smoother. I typically use <a href="https://ffmpeg.org/">ffmpeg</a> and <a href="https://ezgif.com/">ezgif</a> for the finishing touches.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>This timelapse gives a play-by-play of one of the most important campaigns in the war against the Islamic State. In the first few frames, Mosul is under the control of the Kurdistan Regional Government (KRG). In the summer of 2014, ISIS captures the city, and power is cut off. Mosul and many villages along the Tigris river are plunged into darkness. In 2015, the front line in the campaign to retake the city emerges around Mosul, advancing in 2016 and 2017. Mosul is eventually retaken by the KRG in 2017, after which it brightens once again as electricity is restored.</p>
|
||||
</section>
|
||||
<section id="the-qayyarah-fires" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="the-qayyarah-fires">The Qayyarah Fires</h3>
|
||||
<p>Farther south, there is an interesting detail. Above the “h” in “Qayyarah”, a bright set of lights emerges just before Mosul is recaptured, around December 2016. Fleeing Islamic State fighters <a href="https://time.com/iraq-fires/">set fire to the Qayyarah oilfields</a>, which burned for months.</p>
|
||||
<p>Using the VIIRS data we’ve already loaded, we can further analyze the effect of the conflict using a chart. First, let’s define two rectangles (again, you can draw these) over Mosul and Qayyarah:</p>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> mosul <span class="op">=</span> ee<span class="op">.</span><span class="fu">Feature</span>(</span>
|
||||
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">43.054977780266675</span><span class="op">,</span> <span class="fl">36.438274276521234</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.054977780266675</span><span class="op">,</span> <span class="fl">36.290642221212416</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.24792516796199</span><span class="op">,</span> <span class="fl">36.290642221212416</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.24792516796199</span><span class="op">,</span> <span class="fl">36.438274276521234</span>]]]<span class="op">,</span> <span class="kw">null</span><span class="op">,</span> <span class="kw">false</span>)<span class="op">,</span></span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> <span class="st">"label"</span><span class="op">:</span> <span class="st">"Mosul"</span><span class="op">,</span></span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="st">"system:index"</span><span class="op">:</span> <span class="st">"0"</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> })<span class="op">,</span></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> qayyarah <span class="op">=</span> ee<span class="op">.</span><span class="fu">Feature</span>(</span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">43.08240275545117</span><span class="op">,</span> <span class="fl">35.8925587996721</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.08240275545117</span><span class="op">,</span> <span class="fl">35.77899970860588</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.26642375154492</span><span class="op">,</span> <span class="fl">35.77899970860588</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.26642375154492</span><span class="op">,</span> <span class="fl">35.8925587996721</span>]]]<span class="op">,</span> <span class="kw">null</span><span class="op">,</span> <span class="kw">false</span>)<span class="op">,</span></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> <span class="st">"label"</span><span class="op">:</span> <span class="st">"Qayyarah"</span><span class="op">,</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> <span class="st">"system:index"</span><span class="op">:</span> <span class="st">"0"</span></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a> })</span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a><span class="co">// Let's put these together in a list </span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> regions<span class="op">=</span>[qayyarah<span class="op">,</span> mosul]</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Once we’ve got the rectangles, we can make a chart that will take the mean value of the VIIRS images in each rectangle over time:</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> chart <span class="op">=</span></span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a> ui<span class="op">.</span><span class="at">Chart</span><span class="op">.</span><span class="at">image</span></span>
|
||||
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">seriesByRegion</span>({</span>
|
||||
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">imageCollection</span><span class="op">:</span> VIIRS<span class="op">,</span></span>
|
||||
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">regions</span><span class="op">:</span> regions<span class="op">,</span></span>
|
||||
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">reducer</span><span class="op">:</span> ee<span class="op">.</span><span class="at">Reducer</span><span class="op">.</span><span class="fu">mean</span>()<span class="op">,</span></span>
|
||||
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">seriesProperty</span><span class="op">:</span><span class="st">'label'</span></span>
|
||||
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> })<span class="op">.</span><span class="fu">setOptions</span>({</span>
|
||||
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">title</span><span class="op">:</span> <span class="st">'Nighttime Lights'</span></span>
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(chart)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="../images/qayyarah_chart.png" class="img-fluid"></p>
|
||||
<p>We can clearly see Mosul (the red line) darkening in 2014 as the city is taken by ISIS. During this period the Qayyarah oil fields are, as we might expect, quite dark. All of a sudden in 2016 Qayyarah becomes brighter at night than the city of Mosul ever was, as the oilfields are set on fire. Then, almost exactly when the blaze in Qayyarah is extinguished and the area darkens (i.e. when the blue line falls back to near zero), Mosul brightens once again (i.e. the red line rises) as the city is liberated.</p>
|
||||
|
||||
<!--
|
||||
### The Battle for Aleppo
|
||||
|
||||
|
||||
The images below were taken between 2012 and 2014. Vast swaths of the city darken as neighborhoods are razed by fighting.
|
||||
|
||||
|
||||
<timelapse>
|
||||
|
||||
|
||||
Though this is a trend that can be observed across the country, nowhere is the decline in nightlights more visible than in Aleppo. Below is a comparison of longitudinal trends in nightlights signatures between several cities:
|
||||
|
||||
|
||||
<graph>
|
||||
|
||||
|
||||
The most salient trend is Aleppo plummeting over the course of 2012, and becoming steadily darker over the course of the next four years. Raqqa drops in 2012 as well, but remains in flux until 2017, when the battle to reclaim the city plunges it into near total darkness. Damascus also experiences a dip in 2012, but stabilizes relatively quickly. The Turkish city of Gaziantep -- less than 100km from Aleppo and roughly 1/5th the size -- stands in stark contrast to the Syrian cities, becoming progressively brighter over the entire period.
|
||||
|
||||
|
||||
Another interesting pattern here is the difference in seasonal trends in nightlights. Under normal circumstances in this part of the world, cities become brighter at night during the summer months. Restaurants, bars, and markets stay open later and conduct business outdoors. Gaziantep, which still attracts scores of tourists every year, displays pronounced seasonality. Damascus, the most stable of the three Syrian cities, also maintains a seasonal trend throughout the war. In contrast, both Raqqa and Aleppo maintain extremely low and roughly constant levels of nightlights year-round during the periods following intense fighting.
|
||||
|
||||
|
||||
Reliable economic data for Syria haven't been available for nearly a decade, and assessing the country's recovery is consequently difficult. But subtle indications of economic growth are visible above: all three Syrian cities have been on a steady upward trend since 2017, and beginning to display seasonal variation once again. -->
|
||||
<!-- ### Fighting for Oil
|
||||
|
||||
|
||||
Throughout the war, sudden massive spikes in nightlights signatures can be observed throughout the country. In the center of the map just west of Palmyra, some particularly large spikes occur in 2017:
|
||||
|
||||
|
||||
These flashes of light show gas wells being set on fire, a common form of sabotage carried out by retreating Islamic State fighters. Modified Sentinel-2 imagery of the Hayyan gas field (indicated by the green box above) shows this in greater detail. Substituting the Red band in an RGB image with Near Infrared (NIR) highlights thermal signatures, showing fires burning brightly even during the day.
|
||||
|
||||
|
||||
The large complex on the right is the Hayyan Gas Plant, which produced nearly one third of Syria's electricity. The plant and its associated wells changed hands several times throughout the war, but were under Islamic State control until February 2017. In the video below, Islamic State fighters can be seen rigging the plant with explosives and destroying it on January 8th:
|
||||
|
||||
|
||||
In February, three Russian oil and gas companies (Zarubij Naft, Lukoil and Gazprom Neft) were given restoration, exploration and production rights to the hydrocarbon deposits West of Palmyra. On January 12th, 2017, the Syrian Army's 5th Legion and Russian special forces launched a counterattack known as the "Palmyra offensive", with the aim of retaking several important hydrocarbon deposits including Hayyan.
|
||||
|
||||
|
||||
The timing of well fires aligns closely with a detailed timeline of the campaign.The Near Infrared Sentinel-2 image below shows the layout of the Hayyan Gas Plant and the wells in the Hayyan gas field:
|
||||
|
||||
|
||||
The Syrian Army took the Hayyan gas field on [February 4th](https://www.almasdarnews.com/article/syrian-army-liberates-hayyan-gas-fields-west-palmyra/), and retreating ISIS fighters set fire to wells 1, and 3. However, ISIS managed to briefly retake the Hayyan field on [February 7th](https://www.almasdarnews.com/article/isis-retakes-hayyan-gas-fields-new-bid-expand-west-palmyra/), setting fire to wells 2 and 4. These moments in the Palmyra Offensive are captured in NIR signatures
|
||||
|
||||
|
||||
Interestingly, despite the massive explosion caused by the bombing of the Hayyan Gas Plant, no prolonged thermal anomalies were detected over the area of the plant itself. The well fires, on the other hand, lasted for months. Below is an image of well fire at the Hayyan field taken from this [video](https://www.youtube.com/watch?v=WFe9abYyqK0); based on the nearby infrastructure and date (04/02/2017) of posting, it is likely Well-3.
|
||||
-->
|
||||
|
||||
|
||||
</section>
|
||||
</section>
|
||||
|
||||
</main> <!-- /main -->
|
||||
<script id="quarto-html-after-body" type="application/javascript">
|
||||
window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
const toggleBodyColorMode = (bsSheetEl) => {
|
||||
const mode = bsSheetEl.getAttribute("data-mode");
|
||||
const bodyEl = window.document.querySelector("body");
|
||||
if (mode === "dark") {
|
||||
bodyEl.classList.add("quarto-dark");
|
||||
bodyEl.classList.remove("quarto-light");
|
||||
} else {
|
||||
bodyEl.classList.add("quarto-light");
|
||||
bodyEl.classList.remove("quarto-dark");
|
||||
}
|
||||
}
|
||||
const toggleBodyColorPrimary = () => {
|
||||
const bsSheetEl = window.document.querySelector("link#quarto-bootstrap");
|
||||
if (bsSheetEl) {
|
||||
toggleBodyColorMode(bsSheetEl);
|
||||
}
|
||||
}
|
||||
toggleBodyColorPrimary();
|
||||
const disableStylesheet = (stylesheets) => {
|
||||
for (let i=0; i < stylesheets.length; i++) {
|
||||
const stylesheet = stylesheets[i];
|
||||
stylesheet.rel = 'prefetch';
|
||||
}
|
||||
}
|
||||
const enableStylesheet = (stylesheets) => {
|
||||
for (let i=0; i < stylesheets.length; i++) {
|
||||
const stylesheet = stylesheets[i];
|
||||
stylesheet.rel = 'stylesheet';
|
||||
}
|
||||
}
|
||||
const manageTransitions = (selector, allowTransitions) => {
|
||||
const els = window.document.querySelectorAll(selector);
|
||||
for (let i=0; i < els.length; i++) {
|
||||
const el = els[i];
|
||||
if (allowTransitions) {
|
||||
el.classList.remove('notransition');
|
||||
} else {
|
||||
el.classList.add('notransition');
|
||||
}
|
||||
}
|
||||
}
|
||||
const toggleColorMode = (alternate) => {
|
||||
// Switch the stylesheets
|
||||
const alternateStylesheets = window.document.querySelectorAll('link.quarto-color-scheme.quarto-color-alternate');
|
||||
manageTransitions('#quarto-margin-sidebar .nav-link', false);
|
||||
if (alternate) {
|
||||
enableStylesheet(alternateStylesheets);
|
||||
for (const sheetNode of alternateStylesheets) {
|
||||
if (sheetNode.id === "quarto-bootstrap") {
|
||||
toggleBodyColorMode(sheetNode);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
disableStylesheet(alternateStylesheets);
|
||||
toggleBodyColorPrimary();
|
||||
}
|
||||
manageTransitions('#quarto-margin-sidebar .nav-link', true);
|
||||
// Switch the toggles
|
||||
const toggles = window.document.querySelectorAll('.quarto-color-scheme-toggle');
|
||||
for (let i=0; i < toggles.length; i++) {
|
||||
const toggle = toggles[i];
|
||||
if (toggle) {
|
||||
if (alternate) {
|
||||
toggle.classList.add("alternate");
|
||||
} else {
|
||||
toggle.classList.remove("alternate");
|
||||
}
|
||||
}
|
||||
}
|
||||
// Hack to workaround the fact that safari doesn't
|
||||
// properly recolor the scrollbar when toggling (#1455)
|
||||
if (navigator.userAgent.indexOf('Safari') > 0 && navigator.userAgent.indexOf('Chrome') == -1) {
|
||||
manageTransitions("body", false);
|
||||
window.scrollTo(0, 1);
|
||||
setTimeout(() => {
|
||||
window.scrollTo(0, 0);
|
||||
manageTransitions("body", true);
|
||||
}, 40);
|
||||
}
|
||||
}
|
||||
const isFileUrl = () => {
|
||||
return window.location.protocol === 'file:';
|
||||
}
|
||||
const hasAlternateSentinel = () => {
|
||||
let styleSentinel = getColorSchemeSentinel();
|
||||
if (styleSentinel !== null) {
|
||||
return styleSentinel === "alternate";
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
const setStyleSentinel = (alternate) => {
|
||||
const value = alternate ? "alternate" : "default";
|
||||
if (!isFileUrl()) {
|
||||
window.localStorage.setItem("quarto-color-scheme", value);
|
||||
} else {
|
||||
localAlternateSentinel = value;
|
||||
}
|
||||
}
|
||||
const getColorSchemeSentinel = () => {
|
||||
if (!isFileUrl()) {
|
||||
const storageValue = window.localStorage.getItem("quarto-color-scheme");
|
||||
return storageValue != null ? storageValue : localAlternateSentinel;
|
||||
} else {
|
||||
return localAlternateSentinel;
|
||||
}
|
||||
}
|
||||
let localAlternateSentinel = 'alternate';
|
||||
// Dark / light mode switch
|
||||
window.quartoToggleColorScheme = () => {
|
||||
// Read the current dark / light value
|
||||
let toAlternate = !hasAlternateSentinel();
|
||||
toggleColorMode(toAlternate);
|
||||
setStyleSentinel(toAlternate);
|
||||
};
|
||||
// Ensure there is a toggle, if there isn't float one in the top right
|
||||
if (window.document.querySelector('.quarto-color-scheme-toggle') === null) {
|
||||
const a = window.document.createElement('a');
|
||||
a.classList.add('top-right');
|
||||
a.classList.add('quarto-color-scheme-toggle');
|
||||
a.href = "";
|
||||
a.onclick = function() { try { window.quartoToggleColorScheme(); } catch {} return false; };
|
||||
const i = window.document.createElement("i");
|
||||
i.classList.add('bi');
|
||||
a.appendChild(i);
|
||||
window.document.body.appendChild(a);
|
||||
}
|
||||
// Switch to dark mode if need be
|
||||
if (hasAlternateSentinel()) {
|
||||
toggleColorMode(true);
|
||||
} else {
|
||||
toggleColorMode(false);
|
||||
}
|
||||
const icon = "";
|
||||
const anchorJS = new window.AnchorJS();
|
||||
anchorJS.options = {
|
||||
placement: 'right',
|
||||
icon: icon
|
||||
};
|
||||
anchorJS.add('.anchored');
|
||||
const isCodeAnnotation = (el) => {
|
||||
for (const clz of el.classList) {
|
||||
if (clz.startsWith('code-annotation-')) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
const clipboard = new window.ClipboardJS('.code-copy-button', {
|
||||
text: function(trigger) {
|
||||
const codeEl = trigger.previousElementSibling.cloneNode(true);
|
||||
for (const childEl of codeEl.children) {
|
||||
if (isCodeAnnotation(childEl)) {
|
||||
childEl.remove();
|
||||
}
|
||||
}
|
||||
return codeEl.innerText;
|
||||
}
|
||||
});
|
||||
clipboard.on('success', function(e) {
|
||||
// button target
|
||||
const button = e.trigger;
|
||||
// don't keep focus
|
||||
button.blur();
|
||||
// flash "checked"
|
||||
button.classList.add('code-copy-button-checked');
|
||||
var currentTitle = button.getAttribute("title");
|
||||
button.setAttribute("title", "Copied!");
|
||||
let tooltip;
|
||||
if (window.bootstrap) {
|
||||
button.setAttribute("data-bs-toggle", "tooltip");
|
||||
button.setAttribute("data-bs-placement", "left");
|
||||
button.setAttribute("data-bs-title", "Copied!");
|
||||
tooltip = new bootstrap.Tooltip(button,
|
||||
{ trigger: "manual",
|
||||
customClass: "code-copy-button-tooltip",
|
||||
offset: [0, -8]});
|
||||
tooltip.show();
|
||||
}
|
||||
setTimeout(function() {
|
||||
if (tooltip) {
|
||||
tooltip.hide();
|
||||
button.removeAttribute("data-bs-title");
|
||||
button.removeAttribute("data-bs-toggle");
|
||||
button.removeAttribute("data-bs-placement");
|
||||
}
|
||||
button.setAttribute("title", currentTitle);
|
||||
button.classList.remove('code-copy-button-checked');
|
||||
}, 1000);
|
||||
// clear code selection
|
||||
e.clearSelection();
|
||||
});
|
||||
function tippyHover(el, contentFn) {
|
||||
const config = {
|
||||
allowHTML: true,
|
||||
content: contentFn,
|
||||
maxWidth: 500,
|
||||
delay: 100,
|
||||
arrow: false,
|
||||
appendTo: function(el) {
|
||||
return el.parentElement;
|
||||
},
|
||||
interactive: true,
|
||||
interactiveBorder: 10,
|
||||
theme: 'quarto',
|
||||
placement: 'bottom-start'
|
||||
};
|
||||
window.tippy(el, config);
|
||||
}
|
||||
const noterefs = window.document.querySelectorAll('a[role="doc-noteref"]');
|
||||
for (var i=0; i<noterefs.length; i++) {
|
||||
const ref = noterefs[i];
|
||||
tippyHover(ref, function() {
|
||||
// use id or data attribute instead here
|
||||
let href = ref.getAttribute('data-footnote-href') || ref.getAttribute('href');
|
||||
try { href = new URL(href).hash; } catch {}
|
||||
const id = href.replace(/^#\/?/, "");
|
||||
const note = window.document.getElementById(id);
|
||||
return note.innerHTML;
|
||||
});
|
||||
}
|
||||
let selectedAnnoteEl;
|
||||
const selectorForAnnotation = ( cell, annotation) => {
|
||||
let cellAttr = 'data-code-cell="' + cell + '"';
|
||||
let lineAttr = 'data-code-annotation="' + annotation + '"';
|
||||
const selector = 'span[' + cellAttr + '][' + lineAttr + ']';
|
||||
return selector;
|
||||
}
|
||||
const selectCodeLines = (annoteEl) => {
|
||||
const doc = window.document;
|
||||
const targetCell = annoteEl.getAttribute("data-target-cell");
|
||||
const targetAnnotation = annoteEl.getAttribute("data-target-annotation");
|
||||
const annoteSpan = window.document.querySelector(selectorForAnnotation(targetCell, targetAnnotation));
|
||||
const lines = annoteSpan.getAttribute("data-code-lines").split(",");
|
||||
const lineIds = lines.map((line) => {
|
||||
return targetCell + "-" + line;
|
||||
})
|
||||
let top = null;
|
||||
let height = null;
|
||||
let parent = null;
|
||||
if (lineIds.length > 0) {
|
||||
//compute the position of the single el (top and bottom and make a div)
|
||||
const el = window.document.getElementById(lineIds[0]);
|
||||
top = el.offsetTop;
|
||||
height = el.offsetHeight;
|
||||
parent = el.parentElement.parentElement;
|
||||
if (lineIds.length > 1) {
|
||||
const lastEl = window.document.getElementById(lineIds[lineIds.length - 1]);
|
||||
const bottom = lastEl.offsetTop + lastEl.offsetHeight;
|
||||
height = bottom - top;
|
||||
}
|
||||
if (top !== null && height !== null && parent !== null) {
|
||||
// cook up a div (if necessary) and position it
|
||||
let div = window.document.getElementById("code-annotation-line-highlight");
|
||||
if (div === null) {
|
||||
div = window.document.createElement("div");
|
||||
div.setAttribute("id", "code-annotation-line-highlight");
|
||||
div.style.position = 'absolute';
|
||||
parent.appendChild(div);
|
||||
}
|
||||
div.style.top = top - 2 + "px";
|
||||
div.style.height = height + 4 + "px";
|
||||
let gutterDiv = window.document.getElementById("code-annotation-line-highlight-gutter");
|
||||
if (gutterDiv === null) {
|
||||
gutterDiv = window.document.createElement("div");
|
||||
gutterDiv.setAttribute("id", "code-annotation-line-highlight-gutter");
|
||||
gutterDiv.style.position = 'absolute';
|
||||
const codeCell = window.document.getElementById(targetCell);
|
||||
const gutter = codeCell.querySelector('.code-annotation-gutter');
|
||||
gutter.appendChild(gutterDiv);
|
||||
}
|
||||
gutterDiv.style.top = top - 2 + "px";
|
||||
gutterDiv.style.height = height + 4 + "px";
|
||||
}
|
||||
selectedAnnoteEl = annoteEl;
|
||||
}
|
||||
};
|
||||
const unselectCodeLines = () => {
|
||||
const elementsIds = ["code-annotation-line-highlight", "code-annotation-line-highlight-gutter"];
|
||||
elementsIds.forEach((elId) => {
|
||||
const div = window.document.getElementById(elId);
|
||||
if (div) {
|
||||
div.remove();
|
||||
}
|
||||
});
|
||||
selectedAnnoteEl = undefined;
|
||||
};
|
||||
// Attach click handler to the DT
|
||||
const annoteDls = window.document.querySelectorAll('dt[data-target-cell]');
|
||||
for (const annoteDlNode of annoteDls) {
|
||||
annoteDlNode.addEventListener('click', (event) => {
|
||||
const clickedEl = event.target;
|
||||
if (clickedEl !== selectedAnnoteEl) {
|
||||
unselectCodeLines();
|
||||
const activeEl = window.document.querySelector('dt[data-target-cell].code-annotation-active');
|
||||
if (activeEl) {
|
||||
activeEl.classList.remove('code-annotation-active');
|
||||
}
|
||||
selectCodeLines(clickedEl);
|
||||
clickedEl.classList.add('code-annotation-active');
|
||||
} else {
|
||||
// Unselect the line
|
||||
unselectCodeLines();
|
||||
clickedEl.classList.remove('code-annotation-active');
|
||||
}
|
||||
});
|
||||
}
|
||||
const findCites = (el) => {
|
||||
const parentEl = el.parentElement;
|
||||
if (parentEl) {
|
||||
const cites = parentEl.dataset.cites;
|
||||
if (cites) {
|
||||
return {
|
||||
el,
|
||||
cites: cites.split(' ')
|
||||
};
|
||||
} else {
|
||||
return findCites(el.parentElement)
|
||||
}
|
||||
} else {
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
var bibliorefs = window.document.querySelectorAll('a[role="doc-biblioref"]');
|
||||
for (var i=0; i<bibliorefs.length; i++) {
|
||||
const ref = bibliorefs[i];
|
||||
const citeInfo = findCites(ref);
|
||||
if (citeInfo) {
|
||||
tippyHover(citeInfo.el, function() {
|
||||
var popup = window.document.createElement('div');
|
||||
citeInfo.cites.forEach(function(cite) {
|
||||
var citeDiv = window.document.createElement('div');
|
||||
citeDiv.classList.add('hanging-indent');
|
||||
citeDiv.classList.add('csl-entry');
|
||||
var biblioDiv = window.document.getElementById('ref-' + cite);
|
||||
if (biblioDiv) {
|
||||
citeDiv.innerHTML = biblioDiv.innerHTML;
|
||||
}
|
||||
popup.appendChild(citeDiv);
|
||||
});
|
||||
return popup.innerHTML;
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
</script>
|
||||
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./B4_Vectors_Tables.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Vectors and Tables</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./C2_Refineries.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Refinery Identification</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
</div> <!-- /content -->
|
||||
|
||||
|
||||
|
||||
</body></html>
|
||||
946
docs/C2_Refineries.html
Normal file
946
docs/C2_Refineries.html
Normal file
File diff suppressed because one or more lines are too long
1084
docs/C3_Blast.html
Normal file
1084
docs/C3_Blast.html
Normal file
File diff suppressed because it is too large
Load Diff
1067
docs/C4_Ships.html
Normal file
1067
docs/C4_Ships.html
Normal file
File diff suppressed because it is too large
Load Diff
947
docs/C5_Object_Detection.html
Normal file
947
docs/C5_Object_Detection.html
Normal file
@@ -0,0 +1,947 @@
|
||||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>
|
||||
|
||||
<meta charset="utf-8">
|
||||
<meta name="generator" content="quarto-1.3.326">
|
||||
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - 12 Object Detection</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
div.columns{display: flex; gap: min(4vw, 1.5em);}
|
||||
div.column{flex: auto; overflow-x: auto;}
|
||||
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
||||
ul.task-list{list-style: none;}
|
||||
ul.task-list li input[type="checkbox"] {
|
||||
width: 0.8em;
|
||||
margin: 0 0.8em 0.2em -1em; /* quarto-specific, see https://github.com/quarto-dev/quarto-cli/issues/4556 */
|
||||
vertical-align: middle;
|
||||
}
|
||||
/* CSS for syntax highlighting */
|
||||
pre > code.sourceCode { white-space: pre; position: relative; }
|
||||
pre > code.sourceCode > span { display: inline-block; line-height: 1.25; }
|
||||
pre > code.sourceCode > span:empty { height: 1.2em; }
|
||||
.sourceCode { overflow: visible; }
|
||||
code.sourceCode > span { color: inherit; text-decoration: inherit; }
|
||||
div.sourceCode { margin: 1em 0; }
|
||||
pre.sourceCode { margin: 0; }
|
||||
@media screen {
|
||||
div.sourceCode { overflow: auto; }
|
||||
}
|
||||
@media print {
|
||||
pre > code.sourceCode { white-space: pre-wrap; }
|
||||
pre > code.sourceCode > span { text-indent: -5em; padding-left: 5em; }
|
||||
}
|
||||
pre.numberSource code
|
||||
{ counter-reset: source-line 0; }
|
||||
pre.numberSource code > span
|
||||
{ position: relative; left: -4em; counter-increment: source-line; }
|
||||
pre.numberSource code > span > a:first-child::before
|
||||
{ content: counter(source-line);
|
||||
position: relative; left: -1em; text-align: right; vertical-align: baseline;
|
||||
border: none; display: inline-block;
|
||||
-webkit-touch-callout: none; -webkit-user-select: none;
|
||||
-khtml-user-select: none; -moz-user-select: none;
|
||||
-ms-user-select: none; user-select: none;
|
||||
padding: 0 4px; width: 4em;
|
||||
}
|
||||
pre.numberSource { margin-left: 3em; padding-left: 4px; }
|
||||
div.sourceCode
|
||||
{ }
|
||||
@media screen {
|
||||
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
|
||||
}
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./C4_Ships.html" rel="prev">
|
||||
<link href="./../favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="site_libs/quarto-contrib/videojs/video.min.js"></script>
|
||||
<link href="site_libs/quarto-contrib/videojs/video-js.css" rel="stylesheet">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
"collapse-after": 3,
|
||||
"panel-placement": "start",
|
||||
"type": "textbox",
|
||||
"limit": 20,
|
||||
"language": {
|
||||
"search-no-results-text": "No results",
|
||||
"search-matching-documents-text": "matching documents",
|
||||
"search-copy-link-title": "Copy link to search",
|
||||
"search-hide-matches-text": "Hide additional matches",
|
||||
"search-more-match-text": "more match in this document",
|
||||
"search-more-matches-text": "more matches in this document",
|
||||
"search-clear-button-title": "Clear",
|
||||
"search-detached-cancel-button-title": "Cancel",
|
||||
"search-submit-button-title": "Submit"
|
||||
}
|
||||
}</script>
|
||||
<script async="" src="https://www.googletagmanager.com/gtag/js?id=G-RK9ZLZQ6GL"></script>
|
||||
|
||||
<script type="text/javascript">
|
||||
|
||||
window.dataLayer = window.dataLayer || [];
|
||||
function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</script>
|
||||
|
||||
<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.js" type="text/javascript"></script>
|
||||
|
||||
</head>
|
||||
|
||||
<body class="nav-sidebar floating">
|
||||
|
||||
|
||||
<div id="quarto-search-results"></div>
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav">
|
||||
<div class="container-fluid d-flex">
|
||||
<button type="button" class="quarto-btn-toggle btn" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<i class="bi bi-layout-text-sidebar-reverse"></i>
|
||||
</button>
|
||||
<nav class="quarto-page-breadcrumbs" aria-label="breadcrumb"><ol class="breadcrumb"><li class="breadcrumb-item"><a href="./C1_Lights.html">C. Case Studies</a></li><li class="breadcrumb-item"><a href="./C5_Object_Detection.html"><span class="chapter-number">12</span> <span class="chapter-title">Object Detection</span></a></li></ol></nav>
|
||||
<a class="flex-grow-1" role="button" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
</a>
|
||||
<button type="button" class="btn quarto-search-button" aria-label="Search" onclick="window.quartoOpenSearch();">
|
||||
<i class="bi bi-search"></i>
|
||||
</button>
|
||||
</div>
|
||||
</nav>
|
||||
</header>
|
||||
<!-- content -->
|
||||
<div id="quarto-content" class="quarto-container page-columns page-rows-contents page-layout-article">
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse collapse-horizontal sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="quarto-navigation-tool px-1" aria-label="Source Code"><i class="bi bi-github"></i></a>
|
||||
<div class="dropdown">
|
||||
<a href="" title="Download" id="quarto-navigation-tool-dropdown-0" class="quarto-navigation-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Download"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="quarto-navigation-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="dropdown">
|
||||
<a href="" title="Share" id="quarto-navigation-tool-dropdown-1" class="quarto-navigation-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Share"><i class="bi bi-share"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="quarto-navigation-tool-dropdown-1">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="https://twitter.com/intent/tweet?url=|url|">
|
||||
<i class="bi bi-bi-twitter pe-1"></i>
|
||||
Twitter
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="https://www.facebook.com/sharer/sharer.php?u=|url|">
|
||||
<i class="bi bi-bi-facebook pe-1"></i>
|
||||
Facebook
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<a href="" class="quarto-color-scheme-toggle quarto-navigation-tool px-1" onclick="window.quartoToggleColorScheme(); return false;" title="Toggle dark mode"><i class="bi"></i></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-2 flex-shrink-0 align-items-center">
|
||||
<div class="sidebar-search">
|
||||
<div id="quarto-search" class="" title="Search"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="sidebar-menu-container">
|
||||
<ul class="list-unstyled mt-1">
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-1" aria-expanded="false">
|
||||
<span class="menu-text">A. Introduction</span></a>
|
||||
<a class="sidebar-item-toggle text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-1" aria-expanded="false" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">1</span> <span class="chapter-title">Overview</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">2</span> <span class="chapter-title">Remote Sensing</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">3</span> <span class="chapter-title">Data Acquisition</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-2" aria-expanded="false">
|
||||
<span class="menu-text">B. Google Earth Engine</span></a>
|
||||
<a class="sidebar-item-toggle text-start collapsed" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-2" aria-expanded="false" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B1_Getting_Started.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">4</span> <span class="chapter-title">Getting Started</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">5</span> <span class="chapter-title">Interpreting Images</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B3_Image_Series.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">6</span> <span class="chapter-title">Image Series</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">7</span> <span class="chapter-title">Vectors and Tables</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li class="sidebar-item sidebar-item-section">
|
||||
<div class="sidebar-item-container">
|
||||
<a class="sidebar-item-text sidebar-link text-start" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-3" aria-expanded="true">
|
||||
<span class="menu-text">C. Case Studies</span></a>
|
||||
<a class="sidebar-item-toggle text-start" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar-section-3" aria-expanded="true" aria-label="Toggle section">
|
||||
<i class="bi bi-chevron-right ms-2"></i>
|
||||
</a>
|
||||
</div>
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C1_Lights.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">8</span> <span class="chapter-title">War at Night</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C2_Refineries.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">9</span> <span class="chapter-title">Refinery Identification</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C3_Blast.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">10</span> <span class="chapter-title">Blast Damage Assessment</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C4_Ships.html" class="sidebar-item-text sidebar-link">
|
||||
<span class="menu-text"><span class="chapter-number">11</span> <span class="chapter-title">Ship Detection</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./C5_Object_Detection.html" class="sidebar-item-text sidebar-link active">
|
||||
<span class="menu-text"><span class="chapter-number">12</span> <span class="chapter-title">Object Detection</span></span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</nav>
|
||||
<div id="quarto-sidebar-glass" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar,#quarto-sidebar-glass"></div>
|
||||
<!-- margin-sidebar -->
|
||||
<div id="quarto-margin-sidebar" class="sidebar margin-sidebar">
|
||||
<nav id="TOC" role="doc-toc" class="toc-active">
|
||||
<h2 id="toc-title">Table of contents</h2>
|
||||
|
||||
<ul>
|
||||
<li><a href="#object-detection-in-satellite-imagery" id="toc-object-detection-in-satellite-imagery" class="nav-link active" data-scroll-target="#object-detection-in-satellite-imagery"><span class="header-section-number">12.1</span> Object Detection in Satellite Imagery</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#yolov5" id="toc-yolov5" class="nav-link" data-scroll-target="#yolov5"><span class="header-section-number">12.1.1</span> YOLOv5</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#training" id="toc-training" class="nav-link" data-scroll-target="#training"><span class="header-section-number">12.2</span> Training</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#accuracy-assessment" id="toc-accuracy-assessment" class="nav-link" data-scroll-target="#accuracy-assessment"><span class="header-section-number">12.2.1</span> Accuracy Assessment</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#inference" id="toc-inference" class="nav-link" data-scroll-target="#inference"><span class="header-section-number">12.3</span> Inference</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#loading-a-trained-model" id="toc-loading-a-trained-model" class="nav-link" data-scroll-target="#loading-a-trained-model"><span class="header-section-number">12.3.1</span> 1. Loading a trained model</a></li>
|
||||
<li><a href="#loading-the-input-imagery" id="toc-loading-the-input-imagery" class="nav-link" data-scroll-target="#loading-the-input-imagery"><span class="header-section-number">12.3.2</span> 2. Loading the input imagery</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/C5_Object_Detection.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content page-columns page-full" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title"><span class="chapter-number">12</span> <span class="chapter-title">Object Detection</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="quarto-title-meta">
|
||||
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
</header>
|
||||
|
||||
<p>The Ship Detection tutorial explored a use case in which we might want to monitor the activity of ships in a particular location. That was a fairly straightforward task: the sea is very flat, and ships (especially large cargo and military vessels) protrude significantly. Using radar imagery, we could just set a threshold because if anything on the water is reflecting radio waves, it’s probably a ship.</p>
|
||||
<p>One shortcoming of this approach is that it doesn’t tell us what <em>kind</em> of ship we’ve detected. Sure, you could use the shape and size to distinguish between a fishing vessel and an aircraft carrier. But what about ships of similar sizes? Or what if you wanted to use satellite imagery to identify things other than ships, like airplanes, cars, or bridges? This sort of task– called <strong>“object detection”</strong> is a bit more complicated.</p>
|
||||
<p>In this tutorial, we’ll be using a deep learning model called <strong>YOLOv5</strong> to detect objects in satellite imagery. We’ll be training the model on a custom dataset, and then using it to dynamically identify objects in satellite imagery of different resolutions pulled from Google Earth Engine. The tutorial is broken up into three sections:</p>
|
||||
<ol type="1">
|
||||
<li>Object detection in satellite imagery<br>
|
||||
</li>
|
||||
<li>Training a deep learning model on a custom dataset</li>
|
||||
<li>Dynamic inference using Google Earth Engine</li>
|
||||
</ol>
|
||||
<p>Unlike previous tutorials which used the GEE JavaScript API, <strong>this one will utilize Python</strong>; this is because these sorts of deep learning models aren’t available in GEE natively yet. By the end, we’ll be able to generate images such as the one below:</p>
|
||||
<div class="column-screen">
|
||||
<p><img src="images/obj_det2.jpg" class="img-fluid"></p>
|
||||
</div>
|
||||
<section id="object-detection-in-satellite-imagery" class="level2" data-number="12.1">
|
||||
<h2 data-number="12.1" class="anchored" data-anchor-id="object-detection-in-satellite-imagery"><span class="header-section-number">12.1</span> Object Detection in Satellite Imagery</h2>
|
||||
<p>Object detection in satellite imagery has a variety of useful applications.</p>
|
||||
<p>There’s the needle-in-a-haystack problem of needing to monitor a large area for a small number of objects. Immediately prior to the invasion of Ukraine, for example, a number of articles emerged showing Russian military vehicles and equipment popping up in small clearings in the forest near the border with Ukraine. Many of these deployments were spotted by painstakingly combing through high resolution satellite imagery, looking for things that look like trucks. One problem with this approach is that you need to know roughly where to look. The second, and more serious problem, is that you need to be on the lookout in the first place. Object detection, applied to satellite imagery, can automatically comb through vast areas and identify objects of interest. If planes and trucks start showing up in unexpected places, you’ll know about it.</p>
|
||||
<p>Perhaps you’re not monitoring that large of an area, but you want frequent updates about what’s going on. What sorts of objects (planes, trucks, cars, etc.) are present? How many of each? Where are they located? Instead of having to manually look through new imagery as it becomes available, you could have a model automatically analyze new collections and output a summary.</p>
|
||||
<section id="yolov5" class="level3" data-number="12.1.1">
|
||||
<h3 data-number="12.1.1" class="anchored" data-anchor-id="yolov5"><span class="header-section-number">12.1.1</span> YOLOv5</h3>
|
||||
<p>Object detection is a fairly complicated task, and there are a number of different approaches to it. In this tutorial, we’ll be using a model called <strong>YOLOv5</strong>. YOLO stands for <strong>You Only Look Once</strong>, and it’s a model that was developed by <a href="https://pjreddie.com/">Joseph Redmon</a> et. al., and the full paper detailing the model can be found <a href="https://arxiv.org/abs/1506.02640">here</a>.</p>
|
||||
<p>The YOLOv5 model is a <strong>convolutional neural network</strong> (CNN), which is a type of deep learning model. CNNs are very good at identifying patterns in images, particularly in small regions of images. This is important for object detection, because we want to be able to identify objects even if they’re partially obscured by other objects.</p>
|
||||
<p>YOLO works by chopping an image up into a grid, and then predicting the location and size of objects in each grid cell:</p>
|
||||
<p><img src="images/yolo.jpg" class="img-fluid"></p>
|
||||
<p>It learns the locations of these objects by training on a dataset of images in which each object is indicated by a <strong>bounding box</strong>. Then, when it’s shown a new image, it will attempt to predict bounding boxes around the objects in that image. The standard YOLO model is trained on the <a href="https://cocodataset.org/#home">COCO dataset</a>, which contains over 200,000 images of 80 different objects ranging from people to cars to dogs. YOLO models pre-trained on this dataset work great out of the box to detect objects in videos, photographs, and live streams. But the nature of the objects we’re interested in is a bit different.</p>
|
||||
<p>Luckily, we can simply re-train the YOLOv5 model on datasets of labeled satellite imagery. The rest of this tutorial will walk through the process of training YOLOv5 on a custom dataset, and then using it to dynamically identify objects in satellite imagery pulled from Google Earth Engine.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="training" class="level2 page-columns page-full" data-number="12.2">
|
||||
<h2 data-number="12.2" class="anchored" data-anchor-id="training"><span class="header-section-number">12.2</span> Training</h2>
|
||||
<p>The process of re-training the YOLOv5 model on satellite imagery is fairly straightforward and can be accomplished in just three steps; first, we’re going to clone the YOLOv5 repository which contains the model code and the training scripts. Then, we’ll download a dataset of satellite imagery and labels from Roboflow, and finally, we’ll train the model on that dataset.</p>
|
||||
<p>Let’s start by cloning the YOLOv5 repository. Note: we’ll be using a fork of the original repository that I’ve modified to include some pre-trained models that we’ll be using later on.</p>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>git clone https:<span class="op">//</span>github.com<span class="op">/</span>oballinger<span class="op">/</span>yolov5_RS <span class="co"># clone repo</span></span>
|
||||
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="op">%</span>cd yolov5_RS <span class="co"># change directory to repo</span></span>
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="op">%</span>pip install <span class="op">-</span>qr requirements.txt <span class="co"># install dependencies</span></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="op">%</span>pip install <span class="op">-</span>q roboflow <span class="co"># install roboflow</span></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> torch <span class="co"># install pytorch</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> os <span class="co"># for os related operations</span></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> IPython.display <span class="im">import</span> Image, clear_output <span class="co"># to display images</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Once we’ve downloaded the YOLOv5 repository, we’ll need to download a dataset of labeled satellite imagery. For this example, we’re going to stick with ship detection as our use case, but expand upon it. We want to be able to distinguish between different types of ships, and we want to use freely-available satellite imagery.</p>
|
||||
<p>To that end, we’ll be using <a href="https://universe.roboflow.com/ibl-huczk/ships-2fvbx">this dataset</a>, which contains 3400 labeled images taken from Sentinel-2 (10m/px) and PlanetScope (3m/px) satellites. Ships in these images are labeled by drawing an outline around them:</p>
|
||||
<p><img src="images/sample_training_ships.jpg" class="img-fluid"></p>
|
||||
<p>The image above shows three ships and what is known as an STS – a “Ship-To-Ship” transfer – which is when a ship is transferring cargo to another ship. There are a total of seven classes of ship in this dataset:</p>
|
||||
<p><img src="images/label_freq.jpg" class="img-fluid"></p>
|
||||
<p>This dataset can be downloaded directly from Roboflow using the following code:</p>
|
||||
<div class="sourceCode" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> roboflow <span class="im">import</span> Roboflow</span>
|
||||
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a>rf <span class="op">=</span> Roboflow(api_key<span class="op">=</span><span class="st">"<YOUR API KEY>"</span>)</span>
|
||||
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a>project <span class="op">=</span> rf.workspace(<span class="st">'ibl-huczk'</span>).project(<span class="st">"ships-2fvbx"</span>)</span>
|
||||
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a>dataset <span class="op">=</span> project.version(<span class="st">"1"</span>).download(<span class="st">"yolov5"</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>You’ll need to get your own API key from Roboflow, which you can do <a href="https://app.roboflow.com/account/api">here</a>, and insert it in the second line of code. Roboflow is a platform for managing and training deep learning models on custom datasets. It’s free to use for up to three projects, and hosts a large number of datasets that you can use to train your models. To use a different dataset, you can simply change the project name and version number in the second and third lines of code.</p>
|
||||
<p>Finally, we can train our YOLOv5 model on the dataset we just downloaded in just one line of code:</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>python train.py <span class="op">--</span>data {dataset.location}<span class="op">/</span>data.yaml <span class="op">--</span>batch <span class="dv">32</span> <span class="op">--</span>cache</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>This should take about an hour.</p>
|
||||
<section id="accuracy-assessment" class="level3 page-columns page-full" data-number="12.2.1">
|
||||
<h3 data-number="12.2.1" class="anchored" data-anchor-id="accuracy-assessment"><span class="header-section-number">12.2.1</span> Accuracy Assessment</h3>
|
||||
<p>Using Tensorboard, we can log the performance of our model over the course of the training process:</p>
|
||||
<div class="column-page">
|
||||
<iframe src="https://tensorboard.dev/experiment/Yiyl7AsoQcyJ3uw699CR8A/#scalars" width="100%" height="700px">
|
||||
</iframe>
|
||||
</div>
|
||||
<p>One metric in particular, <strong>mAP 0.5</strong>, is a good indicator of how well our model is performing. We can see it increasing rapidly at first, and then leveling off after around 30 epochs of training. The rest of this subsection will explain what exactly the mAP 0.5 value represents in this context. If you’re interested in training your own model at some point, the rest of this subsection will be of interest. If you’re just interested in deploying a pre-trained model, you can skip ahead to the next subsection.</p>
|
||||
<p>In the past when we’ve worked on machine learning projects (for example in the makeshift refinery identifion tutorial), our training and validation data was a set of points – geographic coordinates – which we labeled as either being a refinery or not. Calculating the accuracy of that model was fairly straightforward, since predictions were either true positives, true negatives, false positives or false negatives.</p>
|
||||
<p>This is slightly more complicated for object detection. We’re not going pixel-by-pixel and trying to say “this is a ship” or “this is not a ship.” Instead, we’re looking at a larger image, and trying to draw boxes around the ships. The problem is that there are many ways to draw a box around a ship. The image below shows the labels used in our training data to indicate the location of ships.</p>
|
||||
<p><img src="images/val_batch0_labels.jpg" class="img-fluid"> <img src="images/val_batch0_pred.jpg" class="img-fluid"></p>
|
||||
<p>The predicted bounding boxes are very close to the actual bounding boxes, but they’re not exactly the same. The first step in evaluating the performance of our model is to determine how close the predicted boxes are to the actual boxes. We can do this by calculating the <strong>intersection over union</strong> (IoU) of the predicted and actual boxes. This is essentially a measure of how much overlap there is between the the predicted and actual boxes:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="images/iou.png" height="400" class="figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Intersection over Union</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>The IoU is a value between 0 and 1, where 0 means that the boxes don’t overlap at all, and 1 means that the boxes overlap perfectly. Now we can set a threshold value for the IoU, and say that if the IoU is greater than that threshold, then we’ll count that as a correct prediction. Now that we can classify a prediction as correct or incorrect, we can calculate two important metrics: <span class="math display">\[\text{Precision} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}}\]</span></p>
|
||||
<p>This is the proportion of positive identifications that are actually correct. If my model detects 100 ships and 90 of them are actually ships, then my precision is 90%.</p>
|
||||
<p><span class="math display">\[\text{Recall} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Negatives}}\]</span></p>
|
||||
<p>This is the proportion of actual positives that are identified correctly. If there are 100 ships in the image, and my model detects 90 of them, then my recall is 90%.</p>
|
||||
<p>These two metrics are inversely related; I could easily get 100% recall by drawing lots of boxes everywhere to increase my chances of detecting all the ships. Conversely, I could get 100% precision by being extremely conservative and just drawing one or two boxes around the ships I’m most confident about. The key is to maximize <em>both</em>: we want our model to be sensitive enough to detect as many ships as possible (high recall), but also precise enough to only draw boxes around the ships that are actually there (high precision). Researchers find this balance using a <strong>Precision-Recall curve</strong> (PR curve), which plots precision on the y-axis and recall on the x-axis. Below is the Precision-Recall curve for our final model, for each class:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="images/pr_curve.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Precision-Recall curve from the best</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Starting from the top left corner, we set a very high confidence threshold: precision is 1, meaning that every box we draw is a ship, but recall is near 0 meaning that we’re not detecting any ships. As we lower the confidence threshold, we start to detect more ships, but we also start to draw boxes around things that aren’t ships. Towards the middle of the curve, we’re detecting most of the ships, but we’re also drawing boxes around a lot of false positives. Towards the bottom right corner, we’re detecting all the ships, but we’re also generating lots of false positives.</p>
|
||||
<p>The goal is to find the point on the curve where precision and recall are both high; the closer the peak of our curve is to the top right corner, the better. A perfect model would touch the top right corner: it would have precision of 1 and recall of 1, detecting all of the ships without making any false positives. The area under this curve is called the <strong>Average Precision</strong> (AP), and is a measure of how close the curve is to the top right corner. The perfect model would have an AP of 1.</p>
|
||||
<p>Some classes have a very high AP – the value for the Aircraft Carrier class is 0.995, which is very high (though this could be down to the fact that we have a relatively small number of images with aircraft carriers in them). Ship-To-Ship (STS) transfer operations also have a high AP, at 0.951. However, other classes – notably the “Ship” class – have a low AP. This may be because the “Ship” class is a catch-all for any ship that doesn’t fit into one of the other classes, so it encompasses lots of weird looking ships.</p>
|
||||
<p>Finally, the <strong>mean Average Precision</strong> (mAP) is the average of the AP for each class, shown as the thick blue line above. Remember, all of this is premised on using a 0.5 threshold in the overlap (IoU) between our predicted boxes and the labels, which is why the final metric is called <strong>mAP 0.5</strong>. The mAP 0.5 for our model is 0.775, which is pretty good.</p>
|
||||
<p>This number is very useful when training a model in several different ways using the same dataset, in order to select the best performing one. It’s not that useful for comparing models trained on different datasets, since the mAP 0.5 is dependent on the number of classes in the dataset and the nature of those classes. For example, in the next section we’ll be using a different model trained on the DOTA dataset which has a mAP 0.5 of around 0.68, largely due to the fact that it has around twice as many classes and many of them are similar to each other.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="inference" class="level2 page-columns page-full" data-number="12.3">
|
||||
<h2 data-number="12.3" class="anchored" data-anchor-id="inference"><span class="header-section-number">12.3</span> Inference</h2>
|
||||
<p>Now that we’ve got a trained model, we can use it to conduct object detection on new images. we’ll build a data processing pipeline in three steps by:</p>
|
||||
<ol type="1">
|
||||
<li>Loading our trained model</li>
|
||||
<li>Creating an interactive map to define the area we want to analyze.</li>
|
||||
<li>Defining a function to run object detection within this area.</li>
|
||||
</ol>
|
||||
<section id="loading-a-trained-model" class="level3" data-number="12.3.1">
|
||||
<h3 data-number="12.3.1" class="anchored" data-anchor-id="loading-a-trained-model"><span class="header-section-number">12.3.1</span> 1. Loading a trained model</h3>
|
||||
<p>During the training process, YOLO is iteratively tweaking the model to try to maximize mAP 0.5. It automatically saves the best version of the model in the following location: <code>YOLOv5_RS/runs/train/exp/weights/best.pt</code>. You can save this file for later use, which I have done in case you just want to use this model without having to train it yourself. I’ve also included several other pre-trained models which you can find in the <code>YOLOv5_RS/weights/</code> directory, including:</p>
|
||||
<ul>
|
||||
<li><p><code>lowres_ships.pt</code>: the model we just trained on Sentinel-2 imagery.</p></li>
|
||||
<li><p><code>aircraft.pt</code>: trained on the high resolution <a href="https://www.kaggle.com/datasets/airbusgeo/airbus-aircrafts-sample-dataset">Airbus Aircraft Detection Dataset</a>.</p></li>
|
||||
<li><p><code>general.pt</code>: trained on the <a href="https://captain-whu.github.io/DOTA/dataset.html">DOTA dataset</a> by <a href="https://github.com/KevinMuyaoGuo/yolov5s_for_satellite_imagery#readme">Kevin Guo</a>. This model works great on high resolution satellite imagery, and can detect the following classes: plane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, large vehicle, small vehicle, helicopter, roundabout, soccer field, swimming pool, container crane, airport and helipad.</p></li>
|
||||
</ul>
|
||||
<p>So far, we’ve trained a model to detect ships in Sentinel-2 imagery. But to show the versatility of this general approach, the rest of this tutorial will load up the <code>general.pt</code> model, and use it to detect a wide range of aircraft in high resolution imagery.</p>
|
||||
</section>
|
||||
<section id="loading-the-input-imagery" class="level3 page-columns page-full" data-number="12.3.2">
|
||||
<h3 data-number="12.3.2" class="anchored" data-anchor-id="loading-the-input-imagery"><span class="header-section-number">12.3.2</span> 2. Loading the input imagery</h3>
|
||||
<p>To get started with object detection on satellite imagery using these pre-trained models, we need to define an Area of Interest (AOI) and load satellite imagery. We’ll do this by accessing Google Earth Engine from the Python notebook we’re working in, and creating an interactive map that will let us draw an AOI for analysis.</p>
|
||||
<p>First, we first need to import a few packages:</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>pip install geemap <span class="op">-</span>q</span>
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> pandas <span class="im">as</span> pd</span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> ee</span>
|
||||
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> geemap</span>
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> requests</span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> PIL <span class="im">import</span> Image</span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> PIL <span class="im">import</span> ImageDraw</span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> io <span class="im">import</span> BytesIO</span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> torch</span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> PIL</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Once we’ve done this, we’ll also need to log in to Google Earth Engine using its Python API in order to access the satellite imagery. Running these two lines of code will generate a prompt with instructions; you have to click the link, confirm that you give the notebook permission to access your Earth Engine account, and paste the authentication code in the provided dialogue box.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a>ee.Authenticate()</span>
|
||||
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a>ee.Initialize()</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Great, now we can load high resolution imagery from the National Agriculture Imagery Program (NAIP) and create an interactive map. For this example, I’m centering the map on the <a href="https://en.wikipedia.org/wiki/309th_Aerospace_Maintenance_and_Regeneration_Group">Davis-Monthan Airplane Boneyard</a>. This is where the US Air force retires and restores aircraft, so it will have lots of airplanes of different kinds for us to identify.</p>
|
||||
<p>First, we want to define a function called <code>detect</code> that will accept four arguments:</p>
|
||||
<ul>
|
||||
<li><code>input</code>: the satellite imagery we want to analyze.</li>
|
||||
<li><code>visParams</code>: a dictionary of visualization parameters for the imagery.</li>
|
||||
<li><code>weight</code>: the name of the pre-trained model we want to use.</li>
|
||||
<li><code>labels</code>: a boolean indicating whether we want to display the labels on the processed image.</li>
|
||||
</ul>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> detect(<span class="bu">input</span>, visParams, weight, labels<span class="op">=</span><span class="va">True</span>):</span>
|
||||
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> <span class="co"># Get the AOI from the map</span></span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> aoi <span class="op">=</span> ee.FeatureCollection(Map.draw_features)</span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> mapScale<span class="op">=</span>Map.getScale()</span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="co"># Visualize the raster in Earth Engine and get a download URL</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> image_url<span class="op">=</span><span class="bu">input</span>.visualize(bands<span class="op">=</span>visParams[<span class="st">'bands'</span>], <span class="bu">max</span><span class="op">=</span>visParams[<span class="st">'max'</span>]).getThumbURL({<span class="st">"region"</span>:aoi.geometry(), <span class="st">'scale'</span>:mapScale})</span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> <span class="co"># Load the image into a PIL image</span></span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> response <span class="op">=</span> requests.get(image_url)</span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> img <span class="op">=</span> Image.<span class="bu">open</span>(BytesIO(response.content))</span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> <span class="co"># Load the model</span></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> model <span class="op">=</span>torch.hub.load(<span class="st">'.'</span>,<span class="st">'custom'</span>, path<span class="op">=</span><span class="st">'weights/</span><span class="sc">{}</span><span class="st">.pt'</span>.<span class="bu">format</span>(weight),source<span class="op">=</span><span class="st">'local'</span>,_verbose<span class="op">=</span><span class="va">False</span>)</span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> <span class="co"># Run inference</span></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a> results <span class="op">=</span> model(img)</span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a> <span class="co"># Count the number of detections</span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a> counts<span class="op">=</span>pd.DataFrame(results.pandas().xyxy[<span class="dv">0</span>].groupby(<span class="st">'name'</span>).size()).reset_index().rename(columns<span class="op">=</span>{<span class="dv">0</span>:<span class="st">'count'</span>,<span class="st">'name'</span>:<span class="st">'detected'</span>}).set_index(<span class="st">'count'</span>)</span>
|
||||
<span id="cb6-27"><a href="#cb6-27" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-28"><a href="#cb6-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-29"><a href="#cb6-29" aria-hidden="true" tabindex="-1"></a> <span class="co"># Display the results</span></span>
|
||||
<span id="cb6-30"><a href="#cb6-30" aria-hidden="true" tabindex="-1"></a> results.show(labels<span class="op">=</span>labels)</span>
|
||||
<span id="cb6-31"><a href="#cb6-31" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-32"><a href="#cb6-32" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-33"><a href="#cb6-33" aria-hidden="true" tabindex="-1"></a> <span class="co"># Print the number of detections and the date of the image</span></span>
|
||||
<span id="cb6-34"><a href="#cb6-34" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(ee.Date(<span class="bu">input</span>.get(<span class="st">'system:time_start'</span>)).<span class="bu">format</span>(<span class="st">"dd-MM-yyyy"</span>).getInfo())</span>
|
||||
<span id="cb6-35"><a href="#cb6-35" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(counts)</span>
|
||||
<span id="cb6-36"><a href="#cb6-36" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb6-37"><a href="#cb6-37" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> counts</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now, we can load the NAIP imagery and create an interactive map.</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="co"># load the past 10 years of NAIP imagery</span></span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a>naip <span class="op">=</span> ee.ImageCollection(<span class="st">'USDA/NAIP/DOQQ'</span>).<span class="bu">filter</span>(ee.Filter.date(<span class="st">'2012-01-01'</span>, <span class="st">'2022-01-01'</span>))</span>
|
||||
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a><span class="co"># set some thresholds</span></span>
|
||||
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a>trueColorVis <span class="op">=</span> {</span>
|
||||
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> <span class="st">'bands'</span>:[<span class="st">'R'</span>, <span class="st">'G'</span>, <span class="st">'B'</span>],</span>
|
||||
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> <span class="st">'min'</span>: <span class="dv">0</span>,</span>
|
||||
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a> <span class="st">'max'</span>: <span class="dv">300</span>,</span>
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a><span class="co"># initialize our map</span></span>
|
||||
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a>Map <span class="op">=</span> geemap.Map()</span>
|
||||
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a>Map.setCenter(<span class="op">-</span><span class="fl">110.84</span>,<span class="fl">32.16</span>,<span class="dv">17</span>)</span>
|
||||
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a>Map.addLayer(naip.first(), trueColorVis, <span class="st">"naip"</span>)</span>
|
||||
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a>Map</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>This will generate a small map with some drawing tools on the left side. We can use these tools to draw a polygon around the area we want to analyze. Use the drawing tools to draw a rectangle around an area of interest.</p>
|
||||
<p>Finally, we can run the detection on the imagery. We’ll do this by iterating through the collection of images, and running the <code>detect</code> function on each one. We’ll also store the results in a dataframe so we can analyze them later.</p>
|
||||
<div class="sourceCode" id="cb8"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Get the polygon we just drew on the map </span></span>
|
||||
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a>aoi<span class="op">=</span>ee.FeatureCollection(Map.draw_features)</span>
|
||||
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a><span class="co"># Get a list of all the images in the collection</span></span>
|
||||
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a>naip_list<span class="op">=</span>naip.filterBounds(aoi).toList(naip.size())</span>
|
||||
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a><span class="co"># Iterate through the list of images and run detection on each one</span></span>
|
||||
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> num <span class="kw">in</span> <span class="bu">range</span>(<span class="dv">0</span>,(img_list.size()).getInfo()):</span>
|
||||
<span id="cb8-11"><a href="#cb8-11" aria-hidden="true" tabindex="-1"></a> detect(ee.Image(naip_list.get(num)), trueColorVis,<span class="st">'general'</span>,labels<span class="op">=</span><span class="va">False</span>)</span>
|
||||
<span id="cb8-12"><a href="#cb8-12" aria-hidden="true" tabindex="-1"></a> df<span class="op">=</span>df.append(detection) <span class="co"># store the results in a dataframe</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Below is the result of the detection on the latest image in the collection:</p>
|
||||
<div class="column-screen">
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="images/boneyard.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Davis-Monthan Airplane Boneyard, Tucson AZ. 32.139498, -110.868549</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</div>
|
||||
<p>This image shows a remarkable degree of accuracy being achieved by our model. Inference took just 822.2 milliseconds, and it seems to be doing pretty well. The model identifies over 100 different kinds of aircraft (orange boxes) of many shapes and sizes, civilian and military, without missing a single one. It also identifies around 20 different types of helicopter (blue boxes) in the top right and even spots the cars on the highway and in the parking lots (red boxes). It’s not perfect – it thinks there’s a ship in the bottom left corner near the shed (yellow box); in reality this appears to be half of a plane’s fuselage, an understandable mistake given how long it took <em>me</em> to figure out what it was.</p>
|
||||
<!--
|
||||
Even though we trained our model on Sentinel-2 imagery (10 meters per pixel), it can still be used on imagery from different satellites as long as they have a broadly similar resolution. A ship in PlanetScope imagery (3 meters per pixel) will look roughly similar to a ship in Sentinel-2 imagery. Using PlanetScope has another big advantage over Sentinel-2 beyond its higher spatial resolution: it has a much higher revisit rate (daily instead of 5 days). Though *downloading* PlanetScope imagery isn't free, you *can* generate a timelapse image of any area on Earth using Planet's [Planet Stories](https://www.planet.com/stories/create) tool. Simply create a free account and follow the instructions to generate a timelapse of an area of interest. You can then download the timelapse video and use it as input to our model.
|
||||
|
||||
|
||||
Once you've done this, you can run the following line of code to automatically identify ships in the timelapse video:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
-->
|
||||
|
||||
|
||||
</section>
|
||||
</section>
|
||||
|
||||
</main> <!-- /main -->
|
||||
<script id="quarto-html-after-body" type="application/javascript">
|
||||
window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
const toggleBodyColorMode = (bsSheetEl) => {
|
||||
const mode = bsSheetEl.getAttribute("data-mode");
|
||||
const bodyEl = window.document.querySelector("body");
|
||||
if (mode === "dark") {
|
||||
bodyEl.classList.add("quarto-dark");
|
||||
bodyEl.classList.remove("quarto-light");
|
||||
} else {
|
||||
bodyEl.classList.add("quarto-light");
|
||||
bodyEl.classList.remove("quarto-dark");
|
||||
}
|
||||
}
|
||||
const toggleBodyColorPrimary = () => {
|
||||
const bsSheetEl = window.document.querySelector("link#quarto-bootstrap");
|
||||
if (bsSheetEl) {
|
||||
toggleBodyColorMode(bsSheetEl);
|
||||
}
|
||||
}
|
||||
toggleBodyColorPrimary();
|
||||
const disableStylesheet = (stylesheets) => {
|
||||
for (let i=0; i < stylesheets.length; i++) {
|
||||
const stylesheet = stylesheets[i];
|
||||
stylesheet.rel = 'prefetch';
|
||||
}
|
||||
}
|
||||
const enableStylesheet = (stylesheets) => {
|
||||
for (let i=0; i < stylesheets.length; i++) {
|
||||
const stylesheet = stylesheets[i];
|
||||
stylesheet.rel = 'stylesheet';
|
||||
}
|
||||
}
|
||||
const manageTransitions = (selector, allowTransitions) => {
|
||||
const els = window.document.querySelectorAll(selector);
|
||||
for (let i=0; i < els.length; i++) {
|
||||
const el = els[i];
|
||||
if (allowTransitions) {
|
||||
el.classList.remove('notransition');
|
||||
} else {
|
||||
el.classList.add('notransition');
|
||||
}
|
||||
}
|
||||
}
|
||||
const toggleColorMode = (alternate) => {
|
||||
// Switch the stylesheets
|
||||
const alternateStylesheets = window.document.querySelectorAll('link.quarto-color-scheme.quarto-color-alternate');
|
||||
manageTransitions('#quarto-margin-sidebar .nav-link', false);
|
||||
if (alternate) {
|
||||
enableStylesheet(alternateStylesheets);
|
||||
for (const sheetNode of alternateStylesheets) {
|
||||
if (sheetNode.id === "quarto-bootstrap") {
|
||||
toggleBodyColorMode(sheetNode);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
disableStylesheet(alternateStylesheets);
|
||||
toggleBodyColorPrimary();
|
||||
}
|
||||
manageTransitions('#quarto-margin-sidebar .nav-link', true);
|
||||
// Switch the toggles
|
||||
const toggles = window.document.querySelectorAll('.quarto-color-scheme-toggle');
|
||||
for (let i=0; i < toggles.length; i++) {
|
||||
const toggle = toggles[i];
|
||||
if (toggle) {
|
||||
if (alternate) {
|
||||
toggle.classList.add("alternate");
|
||||
} else {
|
||||
toggle.classList.remove("alternate");
|
||||
}
|
||||
}
|
||||
}
|
||||
// Hack to workaround the fact that safari doesn't
|
||||
// properly recolor the scrollbar when toggling (#1455)
|
||||
if (navigator.userAgent.indexOf('Safari') > 0 && navigator.userAgent.indexOf('Chrome') == -1) {
|
||||
manageTransitions("body", false);
|
||||
window.scrollTo(0, 1);
|
||||
setTimeout(() => {
|
||||
window.scrollTo(0, 0);
|
||||
manageTransitions("body", true);
|
||||
}, 40);
|
||||
}
|
||||
}
|
||||
const isFileUrl = () => {
|
||||
return window.location.protocol === 'file:';
|
||||
}
|
||||
const hasAlternateSentinel = () => {
|
||||
let styleSentinel = getColorSchemeSentinel();
|
||||
if (styleSentinel !== null) {
|
||||
return styleSentinel === "alternate";
|
||||
} else {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
const setStyleSentinel = (alternate) => {
|
||||
const value = alternate ? "alternate" : "default";
|
||||
if (!isFileUrl()) {
|
||||
window.localStorage.setItem("quarto-color-scheme", value);
|
||||
} else {
|
||||
localAlternateSentinel = value;
|
||||
}
|
||||
}
|
||||
const getColorSchemeSentinel = () => {
|
||||
if (!isFileUrl()) {
|
||||
const storageValue = window.localStorage.getItem("quarto-color-scheme");
|
||||
return storageValue != null ? storageValue : localAlternateSentinel;
|
||||
} else {
|
||||
return localAlternateSentinel;
|
||||
}
|
||||
}
|
||||
let localAlternateSentinel = 'alternate';
|
||||
// Dark / light mode switch
|
||||
window.quartoToggleColorScheme = () => {
|
||||
// Read the current dark / light value
|
||||
let toAlternate = !hasAlternateSentinel();
|
||||
toggleColorMode(toAlternate);
|
||||
setStyleSentinel(toAlternate);
|
||||
};
|
||||
// Ensure there is a toggle, if there isn't float one in the top right
|
||||
if (window.document.querySelector('.quarto-color-scheme-toggle') === null) {
|
||||
const a = window.document.createElement('a');
|
||||
a.classList.add('top-right');
|
||||
a.classList.add('quarto-color-scheme-toggle');
|
||||
a.href = "";
|
||||
a.onclick = function() { try { window.quartoToggleColorScheme(); } catch {} return false; };
|
||||
const i = window.document.createElement("i");
|
||||
i.classList.add('bi');
|
||||
a.appendChild(i);
|
||||
window.document.body.appendChild(a);
|
||||
}
|
||||
// Switch to dark mode if need be
|
||||
if (hasAlternateSentinel()) {
|
||||
toggleColorMode(true);
|
||||
} else {
|
||||
toggleColorMode(false);
|
||||
}
|
||||
const icon = "";
|
||||
const anchorJS = new window.AnchorJS();
|
||||
anchorJS.options = {
|
||||
placement: 'right',
|
||||
icon: icon
|
||||
};
|
||||
anchorJS.add('.anchored');
|
||||
const isCodeAnnotation = (el) => {
|
||||
for (const clz of el.classList) {
|
||||
if (clz.startsWith('code-annotation-')) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
const clipboard = new window.ClipboardJS('.code-copy-button', {
|
||||
text: function(trigger) {
|
||||
const codeEl = trigger.previousElementSibling.cloneNode(true);
|
||||
for (const childEl of codeEl.children) {
|
||||
if (isCodeAnnotation(childEl)) {
|
||||
childEl.remove();
|
||||
}
|
||||
}
|
||||
return codeEl.innerText;
|
||||
}
|
||||
});
|
||||
clipboard.on('success', function(e) {
|
||||
// button target
|
||||
const button = e.trigger;
|
||||
// don't keep focus
|
||||
button.blur();
|
||||
// flash "checked"
|
||||
button.classList.add('code-copy-button-checked');
|
||||
var currentTitle = button.getAttribute("title");
|
||||
button.setAttribute("title", "Copied!");
|
||||
let tooltip;
|
||||
if (window.bootstrap) {
|
||||
button.setAttribute("data-bs-toggle", "tooltip");
|
||||
button.setAttribute("data-bs-placement", "left");
|
||||
button.setAttribute("data-bs-title", "Copied!");
|
||||
tooltip = new bootstrap.Tooltip(button,
|
||||
{ trigger: "manual",
|
||||
customClass: "code-copy-button-tooltip",
|
||||
offset: [0, -8]});
|
||||
tooltip.show();
|
||||
}
|
||||
setTimeout(function() {
|
||||
if (tooltip) {
|
||||
tooltip.hide();
|
||||
button.removeAttribute("data-bs-title");
|
||||
button.removeAttribute("data-bs-toggle");
|
||||
button.removeAttribute("data-bs-placement");
|
||||
}
|
||||
button.setAttribute("title", currentTitle);
|
||||
button.classList.remove('code-copy-button-checked');
|
||||
}, 1000);
|
||||
// clear code selection
|
||||
e.clearSelection();
|
||||
});
|
||||
function tippyHover(el, contentFn) {
|
||||
const config = {
|
||||
allowHTML: true,
|
||||
content: contentFn,
|
||||
maxWidth: 500,
|
||||
delay: 100,
|
||||
arrow: false,
|
||||
appendTo: function(el) {
|
||||
return el.parentElement;
|
||||
},
|
||||
interactive: true,
|
||||
interactiveBorder: 10,
|
||||
theme: 'quarto',
|
||||
placement: 'bottom-start'
|
||||
};
|
||||
window.tippy(el, config);
|
||||
}
|
||||
const noterefs = window.document.querySelectorAll('a[role="doc-noteref"]');
|
||||
for (var i=0; i<noterefs.length; i++) {
|
||||
const ref = noterefs[i];
|
||||
tippyHover(ref, function() {
|
||||
// use id or data attribute instead here
|
||||
let href = ref.getAttribute('data-footnote-href') || ref.getAttribute('href');
|
||||
try { href = new URL(href).hash; } catch {}
|
||||
const id = href.replace(/^#\/?/, "");
|
||||
const note = window.document.getElementById(id);
|
||||
return note.innerHTML;
|
||||
});
|
||||
}
|
||||
let selectedAnnoteEl;
|
||||
const selectorForAnnotation = ( cell, annotation) => {
|
||||
let cellAttr = 'data-code-cell="' + cell + '"';
|
||||
let lineAttr = 'data-code-annotation="' + annotation + '"';
|
||||
const selector = 'span[' + cellAttr + '][' + lineAttr + ']';
|
||||
return selector;
|
||||
}
|
||||
const selectCodeLines = (annoteEl) => {
|
||||
const doc = window.document;
|
||||
const targetCell = annoteEl.getAttribute("data-target-cell");
|
||||
const targetAnnotation = annoteEl.getAttribute("data-target-annotation");
|
||||
const annoteSpan = window.document.querySelector(selectorForAnnotation(targetCell, targetAnnotation));
|
||||
const lines = annoteSpan.getAttribute("data-code-lines").split(",");
|
||||
const lineIds = lines.map((line) => {
|
||||
return targetCell + "-" + line;
|
||||
})
|
||||
let top = null;
|
||||
let height = null;
|
||||
let parent = null;
|
||||
if (lineIds.length > 0) {
|
||||
//compute the position of the single el (top and bottom and make a div)
|
||||
const el = window.document.getElementById(lineIds[0]);
|
||||
top = el.offsetTop;
|
||||
height = el.offsetHeight;
|
||||
parent = el.parentElement.parentElement;
|
||||
if (lineIds.length > 1) {
|
||||
const lastEl = window.document.getElementById(lineIds[lineIds.length - 1]);
|
||||
const bottom = lastEl.offsetTop + lastEl.offsetHeight;
|
||||
height = bottom - top;
|
||||
}
|
||||
if (top !== null && height !== null && parent !== null) {
|
||||
// cook up a div (if necessary) and position it
|
||||
let div = window.document.getElementById("code-annotation-line-highlight");
|
||||
if (div === null) {
|
||||
div = window.document.createElement("div");
|
||||
div.setAttribute("id", "code-annotation-line-highlight");
|
||||
div.style.position = 'absolute';
|
||||
parent.appendChild(div);
|
||||
}
|
||||
div.style.top = top - 2 + "px";
|
||||
div.style.height = height + 4 + "px";
|
||||
let gutterDiv = window.document.getElementById("code-annotation-line-highlight-gutter");
|
||||
if (gutterDiv === null) {
|
||||
gutterDiv = window.document.createElement("div");
|
||||
gutterDiv.setAttribute("id", "code-annotation-line-highlight-gutter");
|
||||
gutterDiv.style.position = 'absolute';
|
||||
const codeCell = window.document.getElementById(targetCell);
|
||||
const gutter = codeCell.querySelector('.code-annotation-gutter');
|
||||
gutter.appendChild(gutterDiv);
|
||||
}
|
||||
gutterDiv.style.top = top - 2 + "px";
|
||||
gutterDiv.style.height = height + 4 + "px";
|
||||
}
|
||||
selectedAnnoteEl = annoteEl;
|
||||
}
|
||||
};
|
||||
const unselectCodeLines = () => {
|
||||
const elementsIds = ["code-annotation-line-highlight", "code-annotation-line-highlight-gutter"];
|
||||
elementsIds.forEach((elId) => {
|
||||
const div = window.document.getElementById(elId);
|
||||
if (div) {
|
||||
div.remove();
|
||||
}
|
||||
});
|
||||
selectedAnnoteEl = undefined;
|
||||
};
|
||||
// Attach click handler to the DT
|
||||
const annoteDls = window.document.querySelectorAll('dt[data-target-cell]');
|
||||
for (const annoteDlNode of annoteDls) {
|
||||
annoteDlNode.addEventListener('click', (event) => {
|
||||
const clickedEl = event.target;
|
||||
if (clickedEl !== selectedAnnoteEl) {
|
||||
unselectCodeLines();
|
||||
const activeEl = window.document.querySelector('dt[data-target-cell].code-annotation-active');
|
||||
if (activeEl) {
|
||||
activeEl.classList.remove('code-annotation-active');
|
||||
}
|
||||
selectCodeLines(clickedEl);
|
||||
clickedEl.classList.add('code-annotation-active');
|
||||
} else {
|
||||
// Unselect the line
|
||||
unselectCodeLines();
|
||||
clickedEl.classList.remove('code-annotation-active');
|
||||
}
|
||||
});
|
||||
}
|
||||
const findCites = (el) => {
|
||||
const parentEl = el.parentElement;
|
||||
if (parentEl) {
|
||||
const cites = parentEl.dataset.cites;
|
||||
if (cites) {
|
||||
return {
|
||||
el,
|
||||
cites: cites.split(' ')
|
||||
};
|
||||
} else {
|
||||
return findCites(el.parentElement)
|
||||
}
|
||||
} else {
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
var bibliorefs = window.document.querySelectorAll('a[role="doc-biblioref"]');
|
||||
for (var i=0; i<bibliorefs.length; i++) {
|
||||
const ref = bibliorefs[i];
|
||||
const citeInfo = findCites(ref);
|
||||
if (citeInfo) {
|
||||
tippyHover(citeInfo.el, function() {
|
||||
var popup = window.document.createElement('div');
|
||||
citeInfo.cites.forEach(function(cite) {
|
||||
var citeDiv = window.document.createElement('div');
|
||||
citeDiv.classList.add('hanging-indent');
|
||||
citeDiv.classList.add('csl-entry');
|
||||
var biblioDiv = window.document.getElementById('ref-' + cite);
|
||||
if (biblioDiv) {
|
||||
citeDiv.innerHTML = biblioDiv.innerHTML;
|
||||
}
|
||||
popup.appendChild(citeDiv);
|
||||
});
|
||||
return popup.innerHTML;
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./C4_Ships.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">11</span> <span class="chapter-title">Ship Detection</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
</div>
|
||||
</nav>
|
||||
</div> <!-- /content -->
|
||||
<script>videojs(video_shortcode_videojs_video1);</script>
|
||||
|
||||
|
||||
|
||||
</body></html>
|
||||
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - Remote Sensing</title>
|
||||
<title>Remote Sensing for OSINT - 1 Remote Sensing</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -23,29 +23,29 @@ ul.task-list li input[type="checkbox"] {
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./ch2.html" rel="next">
|
||||
<link href="./index.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="site_libs/quarto-contrib/videojs/video.min.js"></script>
|
||||
<link href="site_libs/quarto-contrib/videojs/video-js.css" rel="stylesheet">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/A3_Data_Acquisition.html" rel="next">
|
||||
<link href="../index.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-contrib/videojs/video.min.js"></script>
|
||||
<link href="../site_libs/quarto-contrib/videojs/video-js.css" rel="stylesheet">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -85,7 +85,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title">Remote Sensing</h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Remote Sensing</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -97,24 +97,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -157,17 +157,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link active">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -182,22 +182,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -212,27 +212,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -255,14 +255,14 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</ul></li>
|
||||
<li><a href="#summary" id="toc-summary" class="nav-link" data-scroll-target="#summary">Summary</a></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/ch1.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/A2_Remote_Sensing.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block">Remote Sensing</h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Remote Sensing</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -277,11 +277,11 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
|
||||
</header>
|
||||
|
||||
<p>Before learning how to load, process, and analyze satellite imagery in Google Earth Engine, it will be helpful to know a few basic principles of remote sensing. This section provides a brief overview of some important concepts and terminology that will be used throughout the course, including active and passive sensors; spatial, spectral, and temporal resolution; and orbits.</p>
|
||||
<p>Before learning how to load, process, and analyze satellite imagery in Google Earth Engine, it will be helpful to know a few basic principles of remote sensing. This section provides a brief overview of some important concepts and terminology that will be used throughout the course, including active and passive sensors; spatial, spectral and temporal resolution; and orbits.</p>
|
||||
<section id="active-and-passive-sensors" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="active-and-passive-sensors">Active and Passive Sensors</h2>
|
||||
<p><a href="https://www.sciencedirect.com/topics/medicine-and-dentistry/remote-sensing">Remote sensing</a> is the science of obtaining information about an object or phenomenon without making physical contact with the object. Remote sensing can be done with various types of electromagnetic radiation such as visible, infrared, or microwave. The electromagnetic radiation is either emitted or reflected from the object being sensed. The reflected radiation is then collected by a sensor and processed to obtain information about the object.</p>
|
||||
<p><img src="./images/diagram.png" class="img-fluid"></p>
|
||||
<p><a href="https://www.sciencedirect.com/topics/medicine-and-dentistry/remote-sensing">Remote sensing</a> is the science of obtaining information about an object or phenomenon without making physical contact with the object. Remote sensing can be done with various types of electromagnetic radiation such as visible, infrared or microwave. The electromagnetic radiation is either emitted or reflected from the object being sensed. The reflected radiation is then collected by a sensor and processed to obtain information about the object.</p>
|
||||
<p><img src="../images/diagram.png" class="img-fluid"></p>
|
||||
<p>While most satellite imagery is optical, meaning it captures sunlight reflected by the earth’s surface, Synthetic Aperture Radar (SAR) satellites such as Sentinel-1 work by emitting pulses of radio waves and measuring how much of the signal is reflected back. This is similar to the way a bat uses sonar to “see” in the dark: by emitting calls and listening to echoes.</p>
|
||||
</section>
|
||||
<section id="resolution" class="level2">
|
||||
@@ -289,8 +289,8 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p>Resolution is one of the most important attributes of satellite imagery. There are three types of resolution: spatial, spectral, and temporal. Let’s look at each of these.</p>
|
||||
<section id="spatial-resolution" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="spatial-resolution">Spatial Resolution</h3>
|
||||
<p>Spatial resolution governs how “sharp” an image looks. The Google Maps satellite basemap, for example, is really sharp Most of the optical imagery that is freely available has relatively low spatial resolution (it looks more grainy than, for example, the Google satellite basemap),</p>
|
||||
<p><img src="./images/Landsat.png" class="img-fluid"> <img src="./images/Sentinel2.png" class="img-fluid"> <img src="./images/Maxar.png" class="img-fluid"></p>
|
||||
<p>Spatial resolution governs how “sharp” an image looks. The Google Maps satellite basemap, for example, is really sharp. Most of the optical imagery that is freely available has relatively low spatial resolution (it looks more grainy than, for example, the Google satellite basemap),</p>
|
||||
<p><img src="../images/Landsat.png" class="img-fluid"> <img src="../images/Sentinel2.png" class="img-fluid"> <img src="../images/Maxar.png" class="img-fluid"></p>
|
||||
</section>
|
||||
<section id="spectral-resolution" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="spectral-resolution">Spectral Resolution</h3>
|
||||
@@ -300,17 +300,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</iframe>
|
||||
<script type="text/javascript">!function(){"use strict";window.addEventListener("message",(function(e){if(void 0!==e.data["datawrapper-height"]){var t=document.querySelectorAll("iframe");for(var a in e.data["datawrapper-height"])for(var r=0;r<t.length;r++){if(t[r].contentWindow===e.source)t[r].style.height=e.data["datawrapper-height"][a]+"px"}}}))}();
|
||||
</script>
|
||||
<p>The visible portion of the spectrum is highlighted on the left, ranging from 400nm (violet) to 700nm (red). Our eyes (and satellite imagery in the visible light spectrum) can only see this portion of the light spectrum; we can’t see UV or infrared wavelengths, for example, though the extent to which different materials reflect or absorb these wavelengths is just as useful for distinguishing between them. The European Space Agency’s Sentinel-2 satellite collects spectral information well beyond the visible light spectrum, enabling this sort of analysis. It chops the electromagnetic spectrum up into “bands”, and measures how strongly wavelengths in each of those bands is reflected:</p>
|
||||
<p>The visible portion of the spectrum is highlighted on the left, ranging from 400 nanometers (violet) to 700nm (red). Our eyes (and satellite imagery in the visible light spectrum) can only see this portion of the light spectrum; we can’t see UV or infrared wavelengths, for example, though the extent to which different materials reflect or absorb these wavelengths is just as useful for distinguishing between them. The European Space Agency’s Sentinel-2 satellite collects spectral information well beyond the visible light spectrum, enabling this sort of analysis. It chops the electromagnetic spectrum up into “bands”, and measures how strongly wavelengths in each of those bands is reflected:</p>
|
||||
<p><img src="images/S2_bands.png" class="img-fluid"></p>
|
||||
<p>To illustrate why this is important, consider Astroturf (fake plastic grass). Astroturf and real grass will both look green to us, espeically from a satellite image. But living plants strongly reflect radiation from the sun in a part of the light spectrum that we can’t see (near-infrared). There’s a spectral index called the Normalized Difference Vegetation Index (NDVI) which exploits this fact to isolate vegetation in multispectral satellite imagery. So if we look at <a href="https://en.wikipedia.org/wiki/Gillette_Stadium">Gilette Stadium</a> near Boston, we can tell that the three training fields south of the stadium are real grass (they generate high NDVI values, showing up red), while the pitch in the stadium itself is astroturf (generating low NDVI values, showing up blue).</p>
|
||||
<p>To illustrate why this is important, consider Astroturf (fake plastic grass). Astroturf and real grass will both look green to us, especially from a satellite image. But living plants strongly reflect radiation from the sun in a part of the light spectrum that we can’t see (near-infrared). There’s a spectral index called the Normalized Difference Vegetation Index (NDVI) which exploits this fact to isolate vegetation in multispectral satellite imagery. So if we look at <a href="https://en.wikipedia.org/wiki/Gillette_Stadium">Gillette Stadium</a> near Boston, we can tell that the three training fields south of the stadium are real grass (they generate high NDVI values, showing up red), while the pitch in the stadium itself is astroturf (generating low NDVI values, showing up blue).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="images/NDVI.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">VHR image of Gilette Stadium with Sentinel-2 derived NDVI overlay</figcaption><p></p>
|
||||
<p></p><figcaption class="figure-caption">VHR image of Gillette Stadium with Sentinel-2 derived NDVI overlay</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>In other words, even though these fields are all green and indistinguishable to the human eye, their <em>spectral profiles</em> beyond the visible light spectrum differ, and we can use this information to distinguish between them.</p>
|
||||
<p>Astroturf is a trivial example. But suppose we were interested in identifying makeshift oil refineries in Northern Syria that constitute a key source of rents for whichever group controls them. As demonstrated in the <a href="./refineries.html">‘Refinery Identification’</a> case study, we can train an algorithm to identify the spectral signatures of oil, and use that to automatically detect them in satellite imagery.</p>
|
||||
<p>Astroturf is a trivial example. But suppose we were interested in identifying makeshift oil refineries in Northern Syria that constitute a key source of rents for whichever group controls them. As demonstrated in the <a href="../chapters/C2_Refineries.html">‘Refinery Identification’</a> case study, we can train an algorithm to identify the spectral signatures of oil, and use that to automatically detect them in satellite imagery.</p>
|
||||
</section>
|
||||
<section id="temporal-resolution" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="temporal-resolution">Temporal Resolution</h3>
|
||||
@@ -318,7 +318,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p>The Google Maps basemap is very high resolution, available globally, and is freely available. But it has no <em>temporal</em> dimension: it’s a snapshot from one particular point in time. If the thing we’re interested in involves <em>changes</em> over time, this basemap will be of limited use.</p>
|
||||
<p>The <strong>“revisit rate”</strong> is the amount of time it takes for the satellite to pass over the same location twice. For example, the Sentinel-2 constellation’s two satellites can achieve a revisit rate of 5 days, as shown in this cool video from the European Space Agency:</p>
|
||||
<div class="quarto-video"><video id="video_shortcode_videojs_video1" class="video-js vjs-default-skin vjs-fluid" controls="" preload="auto" data-setup="{}" title=""><source src="https://dlmultimedia.esa.int/download/public/videos/2016/08/004/1608_004_AR_EN.mp4"></video></div>
|
||||
<p>Some satellite constellations are able to achieve much higher revisit rates. Sentinel-2 has a revisit rate of 5 days, but SkySat capable of imaging the same point on earth around 12 times per day! How is that possible? Well, as the video above demonstrated, the Sentinel-2 constellation is composed of two satellites that share the same orbit, 180 degrees apart. In contrast, the SkySat constellation comprises 21 satellites, each with its own orbital path:</p>
|
||||
<p>Some satellite constellations are able to achieve much higher revisit rates. Sentinel-2 has a revisit rate of 5 days, but SkySat is capable of imaging the same point on earth around 12 times per day! How is that possible? Well, as the video above demonstrated, the Sentinel-2 constellation is composed of two satellites that share the same orbit, 180 degrees apart. In contrast, the SkySat constellation comprises 21 satellites, each with its own orbital path:</p>
|
||||
<div class="quarto-video"><video id="video_shortcode_videojs_video2" class="video-js vjs-default-skin vjs-fluid" controls="" preload="auto" data-setup="{}" title=""><source src="https://assets.planet.com/products/hi-res/Planet_Block_3_HD_1080p.mp4"></video></div>
|
||||
<p>This allows SkySat to achieve a revisit rate of 2-3 <em>hours</em>. The catch, however, is that you need to pay for it (and it <a href="https://apollomapping.com/blog/an-update-on-skysat-tasking-pricing-and-video-capabilities">ain’t cheap</a>). Below is a comparison of revisit rates for various other optical satellites:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
@@ -331,7 +331,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</section>
|
||||
<section id="summary" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="summary">Summary</h2>
|
||||
<p>You should hopefully have a better understanding of what satellite imagery is, and how it can be used to answer questions about the world. In the <a href="./ch2.html">next section</a>, we’ll look at the various types of satellite imagery stored in the Google Earth Engine catalogue.</p>
|
||||
<p>You should hopefully have a better understanding of what satellite imagery is, and how it can be used to answer questions about the world. In the <a href="../chapters/A3_Data_Acquisition.html">next section</a>, we’ll look at the various types of satellite imagery stored in the Google Earth Engine catalog.</p>
|
||||
|
||||
|
||||
</section>
|
||||
@@ -587,13 +587,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./index.html" class="pagination-link">
|
||||
<a href="../index.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text">Overview</span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./ch2.html" class="pagination-link">
|
||||
<span class="nav-page-text">Data Acquisition</span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Data Acquisition</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - Data Acquisition</title>
|
||||
<title>Remote Sensing for OSINT - 2 Data Acquisition</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -23,27 +23,27 @@ ul.task-list li input[type="checkbox"] {
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./F1.html" rel="next">
|
||||
<link href="./ch1.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/B1_Getting_Started.html" rel="next">
|
||||
<link href="../chapters/A2_Remote_Sensing.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -72,9 +72,9 @@ function gtag(){dataLayer.push(arguments);}
|
||||
gtag('js', new Date());
|
||||
gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</script>
|
||||
<script src="site_libs/quarto-diagram/mermaid.min.js"></script>
|
||||
<script src="site_libs/quarto-diagram/mermaid-init.js"></script>
|
||||
<link href="site_libs/quarto-diagram/mermaid.css" rel="stylesheet">
|
||||
<script src="../site_libs/quarto-diagram/mermaid.min.js"></script>
|
||||
<script src="../site_libs/quarto-diagram/mermaid-init.js"></script>
|
||||
<link href="../site_libs/quarto-diagram/mermaid.css" rel="stylesheet">
|
||||
|
||||
|
||||
</head>
|
||||
@@ -86,7 +86,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title">Data Acquisition</h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Data Acquisition</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -98,24 +98,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -158,17 +158,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link active">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -183,22 +183,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -213,27 +213,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -298,14 +298,14 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<li><a href="#datasets-9" id="toc-datasets-9" class="nav-link" data-scroll-target="#datasets-9">Datasets</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/ch2.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/A3_Data_Acquisition.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block">Data Acquisition</h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Data Acquisition</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -320,23 +320,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
|
||||
</header>
|
||||
|
||||
<p>One of the main advantages of GEE is that it hosts several Petabytes of satellite imagery and other spatial data sets, <a href="https://developers.google.com/earth-engine/datasets">all in one place</a>. Among these are a many that could prove useful to those investigating illegal mining and logging, estimating conflict-induced damage, monitoring pollution from extractive industries, conducting maritime surveillance without relying on ship transponders, verifying the locations of artillery strikes, tracking missile defense systems, and many other topics.</p>
|
||||
<p>This section highlights ten categories of geospatial data available natively in the GEE catalogue ranging from optical satellite imagery, to atmospheric data, to building footprints. Each sub-section provides an overview of the given data type, suggests potential applications, and lists the corresponding datasets in the GEE catalogue. The datasets listed under each heading are <strong>not</strong> an exhaustive list– there are over 500 in the whole catalogue, and the ones listed in this section are simply the ones with the most immediate relevance to open source investigations. If a particular geospatial dataset you want to work with isn’t hosted in the GEE catalog, you can upload your own data. We’ll cover that in the next section.</p>
|
||||
<p>One of the main advantages of GEE is that it hosts several Petabytes of satellite imagery and other spatial data sets, <a href="https://developers.google.com/earth-engine/datasets">all in one place</a>. Among these are many that could prove useful to those investigating illegal mining and logging, estimating conflict-induced damage, monitoring pollution from extractive industries, conducting maritime surveillance without relying on ship transponders, verifying the locations of artillery strikes, tracking missile defense systems and many other topics.</p>
|
||||
<p>This section highlights ten categories of geospatial data available natively in the GEE catalog, ranging from optical satellite imagery, to atmospheric data, to building footprints. Each sub-section provides an overview of the given data type, suggests potential applications, and lists the corresponding datasets in the GEE catalog. The datasets listed under each heading are <strong>not</strong> an exhaustive list– there are over 500 in the whole catalog, and the ones listed in this section are simply the ones with the most immediate relevance to open source investigations. If a particular geospatial dataset you want to work with isn’t hosted in the GEE catalog, you can upload your own data. We’ll cover that in the next section.</p>
|
||||
<section id="optical-imagery" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="optical-imagery">Optical Imagery</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/obj_det3.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Automatic detection of vehicles using artificial intelligence in high resolution optical imagery. See the <a href="./object_detection.html">object detection</a> tutorial.</figcaption><p></p>
|
||||
<p><img src="../images/obj_det3.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Automatic detection of vehicles using artificial intelligence in high resolution optical imagery. See the <a href="../chapters/C5_Object_Detection.html">object detection</a> tutorial.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Optical satellite imagery is the bread and butter of many open source investiagtions. It would be tough to list off all of the possible use cases, so here’s a handy flowchart:</p>
|
||||
<p>Optical satellite imagery is the bread and butter of many open source investigations. It would be tough to list off all of the possible use cases, so here’s a handy flowchart:</p>
|
||||
<div class="cell">
|
||||
<div class="cell-output-display">
|
||||
<div>
|
||||
<p>
|
||||
</p><pre class="mermaid mermaid-js" data-tooltip-selector="#mermaid-tooltip-1">%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#FFFFFF' ,'primaryBorderColor':'#000000' , 'lineColor':'#009933'}}}%%
|
||||
|
||||
|
||||
flowchart
|
||||
A(Does it happen outside?)
|
||||
A--> B(Yes)
|
||||
@@ -360,8 +361,8 @@ C-->H
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p>This is, of course, a bit of an exaggeration. But if you’re interested in a visible phenomenon that happens outdoors and that isn’t very tiny, chances are an earth-observing satellite has taken a picture of it. What that picture can tell you naturally depends on what you’re interested in learning. For a deeper dive into analyzing optical satellite imagery, see the subsection on <a href="./ch2.html#multispectral-remote-sensing-remote_sensing">multispectral remote sensing.</a>.</p>
|
||||
<p>There are several different types of optical satellite imagery available in the GEE catalogue. The main collections are the Landsat and Sentinel series of satellites, which are operated by NASA and the European Space Agency, respectively. Landsat satellites have been in orbit since 1972, and Sentinel satellites have been in orbit since 2015. Norway’s International Climate and Forest Initiative (NICFI) has also contributed to the GEE catalogue by providing a collection of optical imagery from Planet’s PlanetScope satellites. These are higher resolution (4.7 meters per pixel) than Landsat (30m/px) and Sentinel-2 (10m/px), but are only available for the tropics. Even higher resolution imagery (60cm/px) is available from the GEE catalogue from the National Agriculture Imagery Program, but it is only available for the United States. For more details, see the “Datasets” section below.</p>
|
||||
<p>This is, of course, a bit of an exaggeration. But if you’re interested in a visible phenomenon that happens outdoors and that isn’t very small, chances are an earth-observing satellite has taken a picture of it. What that picture can tell you naturally depends on what you’re interested in learning. For a deeper dive into analyzing optical satellite imagery, see the subsection on <a href="../chapters/A2_Remote_Sensing.html#multispectral-remote-sensing-remote_sensing">multispectral remote sensing.</a>.</p>
|
||||
<p>There are several different types of optical satellite imagery available in the GEE catalog. The main collections are the Landsat and Sentinel series of satellites, which are operated by NASA and the European Space Agency, respectively. Landsat satellites have been in orbit since 1972, and Sentinel satellites have been in orbit since 2015. Norway’s International Climate and Forest Initiative (NICFI) has also contributed to the GEE catalog by providing a collection of optical imagery from Planet’s PlanetScope satellites. These are higher resolution (4.7 meters per pixel) than Landsat (30m/px) and Sentinel-2 (10m/px), but are only available for the tropics. Even higher resolution imagery (60cm/px) is available from the GEE catalog from the National Agriculture Imagery Program, but it is only available for the United States. For more details, see the “Datasets” section below.</p>
|
||||
<section id="applications" class="level3 unnumbered">
|
||||
<h3 class="unnumbered anchored" data-anchor-id="applications">Applications</h3>
|
||||
<ul>
|
||||
@@ -453,7 +454,7 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="radar-imagery">Radar Imagery</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/radar ships.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/radar ships.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Ships and interference from a radar system are visible in Zhuanghe Wan, near North Korea.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -493,12 +494,12 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="nighttime-lights">Nighttime Lights</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/Figure_1.gif" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/Figure_1.gif" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">A timelapse of nighttime lights over Northern Iraq showing the capture and liberation of Mosul by ISIS.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Satellite images of the Earth at night a useful proxy for human activity. The brightness of a given area at night is a function of the number of people living there and the nature of their activities. The effects of conflict, natural disasters, and economic development can all be inferred from changes in nighttime lights.</p>
|
||||
<p>The timelapse above reveals a number of interesting things: The capture of Mosul by ISIS in 2014 and the destruction of its infrastructure during the fighting (shown as the city darkening), as well as the liberation of the city by the Iraqi military in 2017 are all visible in nighttime lights. The code to create this gif, as well as a more in-depth tutorial on the uses of nighttime lights, can be found in the <a href="./lights.html">“War at Night”</a> case study.</p>
|
||||
<p>Satellite images of the Earth at night are a useful proxy for human activity. The brightness of a given area at night is a function of the number of people living there and the nature of their activities. The effects of conflict, natural disasters, and economic development can all be inferred from changes in nighttime lights.</p>
|
||||
<p>The timelapse above reveals a number of interesting things: The capture of Mosul by ISIS in 2014 and the destruction of its infrastructure during the fighting (shown as the city darkening), as well as the liberation of the city by the Iraqi military in 2017 are all visible in nighttime lights. The code to create this gif, as well as a more in-depth tutorial on the uses of nighttime lights, can be found in the <a href="../chapters/C1_Lights.html">“War at Night”</a> case study.</p>
|
||||
<section id="applications-2" class="level3 unnumbered">
|
||||
<h3 class="unnumbered anchored" data-anchor-id="applications-2">Applications</h3>
|
||||
<ul>
|
||||
@@ -539,12 +540,12 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="climate-and-atmospheric-data">Climate and Atmospheric Data</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/mishraq_small.gif" class="img-fluid figure-img" style="width:100.0%"></p>
|
||||
<p><img src="../images/mishraq_small.gif" class="img-fluid figure-img" style="width:100.0%"></p>
|
||||
<p></p><figcaption class="figure-caption">Sulphur Dioxide plume resulting from ISIS attack on the Al-Mishraq Sulphur Plant in Iraq</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Climate and atmospheric data can be used to track the effects of conflict on the environment. The European Space Agency’s Sentinel-5p satellites measure the concentration of a number of atmospheric gases, including nitrogen dioxide, methane, and ozone. Measurements are available on a daily basis at a fairly high resolution (1km), allowing for the detection of localized sources of pollution such as oil refineries or power plants. For example, see this <a href="https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/">Bellingcat article</a> in which Wim Zwijnenburg and I trace pollution to specific facilities operated by multinational oil companies in Iraq.</p>
|
||||
<p>The Copernicus Atmosphere Monitoring Service (CAMS) provides similar data at a lower spatial resolution (45km), but measurements are avaialble on an hourly basis. The timelapse above utilizes CAMS data to show a sulphur dioxide plume resulting from an ISIS attack on the Al-Mishraq Sulphur Plant in Iraq. The plant was used to produce sulphuric acid, for use in fertilizers and pesticides. The attack destroyed the plant, causing a fire which burned for a month and released <a href="https://earthobservatory.nasa.gov/images/88994/sulfur-dioxide-spreads-over-iraq">21 kilotons</a> of sulphur dioxide into the atmosphere per day; the largest human-made release of sulphur dioxide in history.</p>
|
||||
<p>Climate and atmospheric data can be used to track the effects of conflict on the environment. The European Space Agency’s Sentinel-5p satellites measure the concentration of a number of atmospheric gasses, including nitrogen dioxide, methane and ozone. Measurements are available on a daily basis at a fairly high resolution (1km), allowing for the detection of localized sources of pollution such as oil refineries or power plants. For example, see this <a href="https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/">Bellingcat article</a> in which Wim Zwijnenburg and I trace pollution to specific facilities operated by multinational oil companies in Iraq.</p>
|
||||
<p>The Copernicus Atmosphere Monitoring Service (CAMS) provides similar data at a lower spatial resolution (45km), but measurements are available on an hourly basis. The timelapse above utilizes CAMS data to show a sulfur dioxide plume resulting from an ISIS attack on the Al-Mishraq Sulphur Plant in Iraq. The plant was used to produce sulphuric acid, for use in fertilizers and pesticides. The attack destroyed the plant, causing a fire which burned for a month and released <a href="https://earthobservatory.nasa.gov/images/88994/sulfur-dioxide-spreads-over-iraq">21 kilotons</a> of sulfur dioxide into the atmosphere per day; the largest human-made release of sulfur dioxide in history.</p>
|
||||
<section id="applications-3" class="level3 unnumbered">
|
||||
<h3 class="unnumbered anchored" data-anchor-id="applications-3">Applications</h3>
|
||||
<ul>
|
||||
@@ -589,7 +590,7 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="mineral-deposits">Mineral Deposits</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/mining.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/mining.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Zinc deposits across Central Africa</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -628,7 +629,7 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="fires">Fires</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/fires.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/fires.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Detected fires over Ukraine since 27/02/2022 showing the frontline of the war</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -677,11 +678,11 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="population-density-estimates">Population Density Estimates</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/pop.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/pop.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Population density estimates around Pyongyang, North Korea</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Sometimes, we may want to get an estimate the population in a specific area to ballpark how many people might be affected by a natural disaster, a counteroffensive, or a missile strike. You can’t really google “what is the population in this rectangle i’ve drawn in Northeastern Syria?” and get a good answer. Luckily, there are several spatial population datasets hosted in GEE that let you do just that. Some, such as WorldPop, provide estimated breakdowns by age and sex as well. However, it is extremely important to bear in mind that these are <strong>estimates</strong>, and will <strong>not</strong> take into account things like conflict-induced displacement. For example, Oak Ridge National Laboratory’s LandScan program has released high-resolution population data for Ukraine, but this pertains to the pre-war population distribution. The war has radically changed this distribution, so these estimates no longer reflect where people <em>are</em>. Still, this dataset could be used to roughly estimate displacement or the number of people who will need new housing.</p>
|
||||
<p>Sometimes, we may want to get an estimate of the population in a specific area to ballpark how many people might be affected by a natural disaster, a counteroffensive or a missile strike. You can’t really Google “what is the population in this rectangle I’ve drawn in Northeastern Syria?” and get a good answer. Luckily, there are several spatial population datasets hosted in GEE that let you do just that. Some, such as WorldPop, provide estimated breakdowns by age and sex as well. However, it is extremely important to bear in mind that these are <strong>estimates</strong>, and will <strong>not</strong> take into account things like conflict-induced displacement. For example, Oak Ridge National Laboratory’s LandScan program has released high-resolution population data for Ukraine, but this pertains to the pre-war population distribution. The war has radically changed this distribution, so these estimates no longer reflect where people <em>are</em>. Still, this dataset could be used to roughly estimate displacement or the number of people who will need new housing.</p>
|
||||
<section id="applications-6" class="level3 unnumbered">
|
||||
<h3 class="unnumbered anchored" data-anchor-id="applications-6">Applications:</h3>
|
||||
<ul>
|
||||
@@ -726,7 +727,7 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="building-footprints">Building Footprints</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/footprints.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/footprints.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Building footprints in Mariupol, Ukraine colored by whether the building is damaged</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -761,11 +762,11 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="administrative-boundaries">Administrative Boundaries</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/fao_gaul.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/fao_gaul.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Second-level administrative boundaries in Yemen</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Spatial analysis often have to aggregate information over a defined area; we may want to assess the total burned area by province in Ukraine, or count the number of Saudi airstrikes by district in Yemen. For that, we need data on these administrative boundaries. GEE hosts several such datasets at the country, province, and district (or equivalent) level.</p>
|
||||
<p>Spatial analysis often has to aggregate information over a defined area; we may want to assess the total burned area by province in Ukraine, or count the number of Saudi airstrikes by district in Yemen. For that, we need data on these administrative boundaries. GEE hosts several such datasets at the country, province, and district (or equivalent) level.</p>
|
||||
<section id="applications-8" class="level3 unnumbered">
|
||||
<h3 class="unnumbered anchored" data-anchor-id="applications-8">Applications</h3>
|
||||
<ul>
|
||||
@@ -799,11 +800,11 @@ C-->H
|
||||
<h2 class="anchored" data-anchor-id="global-power-plant-database">Global Power Plant Database</h2>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/power.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/power.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Power plants in Ukraine colored by type</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>The Global Power Plant Database is a comprehensive, open source database of power plants around the world. It centralizes power plant data to make it easier to navigate, compare and draw insights. Each power plant is geolocated and entries contain information on plant capacity, generation, ownership, and fuel type. As of June 2018, the database includes around 28,500 power plants from 164 countries. The database is curated by the <a href="https://datasets.wri.org/dataset/globalpowerplantdatabase">World Resources Institude (WRI)</a>.</p>
|
||||
<p>The Global Power Plant Database is a comprehensive, open source database of power plants around the world. It centralizes power plant data to make it easier to navigate, compare and draw insights. Each power plant is geolocated and entries contain information on plant capacity, generation, ownership, and fuel type. As of June 2018, the database includes around 28,500 power plants from 164 countries. The database is curated by the <a href="https://datasets.wri.org/dataset/globalpowerplantdatabase">World Resources Institute (WRI)</a>.</p>
|
||||
<section id="applications-9" class="level3 unnumbered">
|
||||
<h3 class="unnumbered anchored" data-anchor-id="applications-9">Applications:</h3>
|
||||
<ul>
|
||||
@@ -1088,13 +1089,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./ch1.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text">Remote Sensing</span>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Remote Sensing</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./F1.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Getting Started</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - 2 Interpreting Images</title>
|
||||
<title>Remote Sensing for OSINT - 4 Interpreting Images</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -86,27 +86,27 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./F4.html" rel="next">
|
||||
<link href="./F1.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/B3_Image_Series.html" rel="next">
|
||||
<link href="../chapters/B1_Getting_Started.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -146,7 +146,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Interpreting Images</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -158,24 +158,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -218,17 +218,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -243,22 +243,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -273,27 +273,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -307,35 +307,35 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<h2 id="toc-title">Table of contents</h2>
|
||||
|
||||
<ul>
|
||||
<li><a href="#image-manipulation-bands-arithmetic-thresholds-and-masks" id="toc-image-manipulation-bands-arithmetic-thresholds-and-masks" class="nav-link active" data-scroll-target="#image-manipulation-bands-arithmetic-thresholds-and-masks"><span class="toc-section-number">2.1</span> Image Manipulation: Bands, Arithmetic, Thresholds, and Masks</a>
|
||||
<li><a href="#image-manipulation-bands-arithmetic-thresholds-and-masks" id="toc-image-manipulation-bands-arithmetic-thresholds-and-masks" class="nav-link active" data-scroll-target="#image-manipulation-bands-arithmetic-thresholds-and-masks">Image Manipulation: Bands, Arithmetic, Thresholds, and Masks</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#band-arithmetic-in-earth-engine" id="toc-band-arithmetic-in-earth-engine" class="nav-link" data-scroll-target="#band-arithmetic-in-earth-engine"><span class="toc-section-number">2.1.1</span> Band Arithmetic in Earth Engine</a></li>
|
||||
<li><a href="#thresholding-masking-and-remapping-images" id="toc-thresholding-masking-and-remapping-images" class="nav-link" data-scroll-target="#thresholding-masking-and-remapping-images"><span class="toc-section-number">2.1.2</span> Thresholding, Masking, and Remapping Images</a></li>
|
||||
<li><a href="#band-arithmetic-in-earth-engine" id="toc-band-arithmetic-in-earth-engine" class="nav-link" data-scroll-target="#band-arithmetic-in-earth-engine">Band Arithmetic in Earth Engine</a></li>
|
||||
<li><a href="#thresholding-masking-and-remapping-images" id="toc-thresholding-masking-and-remapping-images" class="nav-link" data-scroll-target="#thresholding-masking-and-remapping-images">Thresholding, Masking, and Remapping Images</a></li>
|
||||
<li><a href="#conclusion" id="toc-conclusion" class="nav-link" data-scroll-target="#conclusion">Conclusion</a></li>
|
||||
<li><a href="#references" id="toc-references" class="nav-link" data-scroll-target="#references">References</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#interpreting-an-image-classification" id="toc-interpreting-an-image-classification" class="nav-link" data-scroll-target="#interpreting-an-image-classification"><span class="toc-section-number">2.2</span> Interpreting an Image: Classification</a>
|
||||
<li><a href="#interpreting-an-image-classification" id="toc-interpreting-an-image-classification" class="nav-link" data-scroll-target="#interpreting-an-image-classification">Interpreting an Image: Classification</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#supervised-classification" id="toc-supervised-classification" class="nav-link" data-scroll-target="#supervised-classification"><span class="toc-section-number">2.2.1</span> Supervised Classification</a></li>
|
||||
<li><a href="#unsupervised-classification" id="toc-unsupervised-classification" class="nav-link" data-scroll-target="#unsupervised-classification"><span class="toc-section-number">2.2.2</span> Unsupervised Classification</a></li>
|
||||
<li><a href="#supervised-classification" id="toc-supervised-classification" class="nav-link" data-scroll-target="#supervised-classification">Supervised Classification</a></li>
|
||||
<li><a href="#unsupervised-classification" id="toc-unsupervised-classification" class="nav-link" data-scroll-target="#unsupervised-classification">Unsupervised Classification</a></li>
|
||||
<li><a href="#conclusion-1" id="toc-conclusion-1" class="nav-link" data-scroll-target="#conclusion-1">Conclusion</a></li>
|
||||
<li><a href="#references-1" id="toc-references-1" class="nav-link" data-scroll-target="#references-1">References</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#accuracy-assessment-quantifying-classification-quality" id="toc-accuracy-assessment-quantifying-classification-quality" class="nav-link" data-scroll-target="#accuracy-assessment-quantifying-classification-quality"><span class="toc-section-number">2.3</span> Accuracy Assessment: Quantifying Classification Quality</a>
|
||||
<li><a href="#accuracy-assessment-quantifying-classification-quality" id="toc-accuracy-assessment-quantifying-classification-quality" class="nav-link" data-scroll-target="#accuracy-assessment-quantifying-classification-quality">Accuracy Assessment: Quantifying Classification Quality</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#quantifying-classification-accuracy-through-a-confusion-matrix" id="toc-quantifying-classification-accuracy-through-a-confusion-matrix" class="nav-link" data-scroll-target="#quantifying-classification-accuracy-through-a-confusion-matrix"><span class="toc-section-number">2.3.1</span> Quantifying Classification Accuracy Through a Confusion Matrix</a></li>
|
||||
<li><a href="#hyperparameter-tuning" id="toc-hyperparameter-tuning" class="nav-link" data-scroll-target="#hyperparameter-tuning"><span class="toc-section-number">2.3.2</span> Hyperparameter tuning</a></li>
|
||||
<li><a href="#quantifying-classification-accuracy-through-a-confusion-matrix" id="toc-quantifying-classification-accuracy-through-a-confusion-matrix" class="nav-link" data-scroll-target="#quantifying-classification-accuracy-through-a-confusion-matrix">Quantifying Classification Accuracy Through a Confusion Matrix</a></li>
|
||||
<li><a href="#hyperparameter-tuning" id="toc-hyperparameter-tuning" class="nav-link" data-scroll-target="#hyperparameter-tuning">Hyperparameter tuning</a></li>
|
||||
<li><a href="#conclusion-2" id="toc-conclusion-2" class="nav-link" data-scroll-target="#conclusion-2">Conclusion</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/F2.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/B2_Interpreting_Images.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Interpreting Images</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -351,8 +351,8 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</header>
|
||||
|
||||
<p>Now that you know how images are viewed and what kinds of images exist in Earth Engine, how do we manipulate them? To gain the skills of interpreting images, you’ll work with bands, combining values to form indices and masking unwanted pixels. Then, you’ll learn some of the techniques available in Earth Engine for classifying images and interpreting the results.</p>
|
||||
<section id="image-manipulation-bands-arithmetic-thresholds-and-masks" class="level2" data-number="2.1">
|
||||
<h2 data-number="2.1" class="anchored" data-anchor-id="image-manipulation-bands-arithmetic-thresholds-and-masks"><span class="header-section-number">2.1</span> Image Manipulation: Bands, Arithmetic, Thresholds, and Masks</h2>
|
||||
<section id="image-manipulation-bands-arithmetic-thresholds-and-masks" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="image-manipulation-bands-arithmetic-thresholds-and-masks">Image Manipulation: Bands, Arithmetic, Thresholds, and Masks</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -388,32 +388,32 @@ Chapter Information
|
||||
</div>
|
||||
<section id="introduction" class="level3 unlisted unnumbered">
|
||||
<h3 class="unlisted unnumbered anchored" data-anchor-id="introduction">Introduction</h3>
|
||||
<p>Spectral indices are based on the fact that different objects and land covers on the Earth’s surface reflect different amounts of light from the Sun at different wavelengths. In the visible part of the spectrum, for example, a healthy green plant reflects a large amount of green light while absorbing blue and red light—which is why it appears green to our eyes. Light also arrives from the Sun at wavelengths outside what the human eye can see, and there are large differences in reflectances between living and nonliving land covers, and between different types of vegetation, both in the visible and outside the visible wavelengths. We visualized this earlier, in Chaps. F1.1 and F1.3 when we mapped color-infrared images (Fig. F2.0.1).</p>
|
||||
<p>Spectral indices are based on the fact that different objects and land covers on the Earth’s surface reflect different amounts of light from the Sun at different wavelengths. In the visible part of the spectrum, for example, a healthy green plant reflects a large amount of green light while absorbing blue and red light — which is why it appears green to our eyes. Light also arrives from the Sun at wavelengths outside what the human eye can see, and there are large differences in reflectances between living and nonliving land covers, and between different types of vegetation, both in the visible and outside the visible wavelengths. We visualized this earlier, in Chaps. F1.1 and F1.3 when we mapped color-infrared images (Fig. F2.0.1).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image39.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image39.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.1 Mapped color-IR images from multiple satellite sensors that we mapped in Chap. F1.3. The near infrared spectrum is mapped as red, showing where there are high amounts of healthy vegetation.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>If we graph the amount of light (reflectance) at different wavelengths that an object or land cover reflects, we can visualize this more easily (Fig. F2.0.2). For example, look at the reflectance curves for soil and water in the graph below. Soil and water both have relatively low reflectance at wavelengths around 300 nm (ultraviolet and violet light). Conversely, at wavelengths above 700 nm (red and infrared light) soil has relatively high reflectance, while water has very low reflectance. Vegetation, meanwhile, generally reflects large amounts of near infrared light, relative to other land covers.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image32.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image32.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.2 A graph of the amount of reflectance for different objects on the Earth’s surface at different wavelengths in the visible and infrared portions of the electromagnetic spectrum. 1 micrometer (µm) = 1,000 nanometers (nm).</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Spectral indices use math to express how objects reflect light across multiple portions of the spectrum as a single number. Indices combine multiple bands, often with simple operations of subtraction and division, to create a single value across an image that is intended to help to distinguish particular land uses or land covers of interest. Using Fig. F2.0.2, you can imagine which wavelengths might be the most informative for distinguishing among a variety of land covers. We will explore a variety of calculations made from combinations of bands in the following sections.</p>
|
||||
<p>Indices derived from satellite imagery are used as the basis of many remote-sensing analyses. Indices have been used in thousands of applications, from detecting anthropogenic deforestation to examining crop health. For example, the growth of economically important crops such as wheat and cotton can be monitored throughout the growing season: Bare soil reflects more red wavelengths, whereas growing crops reflect more of the near-infrared (NIR) wavelengths. Thus, calculating a ratio of these two bands can help monitor how well crops are growing (Jackson and Huete 1991).</p>
|
||||
</section>
|
||||
<section id="band-arithmetic-in-earth-engine" class="level3" data-number="2.1.1">
|
||||
<h3 data-number="2.1.1" class="anchored" data-anchor-id="band-arithmetic-in-earth-engine"><span class="header-section-number">2.1.1</span> Band Arithmetic in Earth Engine</h3>
|
||||
<section id="band-arithmetic-in-earth-engine" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="band-arithmetic-in-earth-engine">Band Arithmetic in Earth Engine</h3>
|
||||
<p>If you have not already done so, be sure to add the book’s code repository to the Code Editor by entering <a href="https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo%3Dprojects/gee-edu/book&sa=D&source=editors&ust=1671458829783542&usg=AOvVaw2f8xfEZP6c0zP_Ke8jL26U"></a><a href="https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo%3Dprojects/gee-edu/book&sa=D&source=editors&ust=1671458829783919&usg=AOvVaw2i09J44MzpMZkjV_JLEnNR">https://code.earthengine.google.com/?accept_repo=projects/gee-edu/book</a> into your browser. The book’s scripts will then be available in the script manager panel. If you have trouble finding the repo, you can visit <a href="https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit%23slide%3Did.g18a7b4b055d_0_624&sa=D&source=editors&ust=1671458829784270&usg=AOvVaw1Kr82KG60ZeFLYC8cOZ67A">this link</a> for help.</p>
|
||||
<p>Many indices can be calculated using band arithmetic in Earth Engine. Band arithmetic is the process of adding, subtracting, multiplying, or dividing two or more bands from an image. Here we’ll first do this manually, and then show you some more efficient ways to perform band arithmetic in Earth Engine.</p>
|
||||
<section id="arithmetic-calculation-of-ndvi" class="level4" data-number="2.1.1.1">
|
||||
<h4 data-number="2.1.1.1" class="anchored" data-anchor-id="arithmetic-calculation-of-ndvi"><span class="header-section-number">2.1.1.1</span> Arithmetic Calculation of NDVI</h4>
|
||||
<section id="arithmetic-calculation-of-ndvi" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="arithmetic-calculation-of-ndvi">Arithmetic Calculation of NDVI</h4>
|
||||
<p>The red and near-infrared bands provide a lot of information about vegetation due to vegetation’s high reflectance in these wavelengths. Take a look at Fig. F2.0.2 and note, in particular, that vegetation curves (graphed in green) have relatively high reflectance in the NIR range (approximately 750–900 nm). Also note that vegetation has low reflectance in the red range (approximately 630–690 nm), where sunlight is absorbed by chlorophyll. This suggests that if the red and near-infrared bands could be combined, they would provide substantial information about vegetation.</p>
|
||||
<p>Soon after the launch of Landsat 1 in 1972, analysts worked to devise a robust single value that would convey the health of vegetation along a scale of −1 to 1. This yielded the NDVI, using the formula:</p>
|
||||
<p><img src="F2/image1.png" class="img-fluid"> (F2.0.1)</p>
|
||||
<p><img src="../images/F2/image1.png" class="img-fluid"> (F2.0.1)</p>
|
||||
<p>where NIR and red refer to the brightness of each of those two bands. As seen in Chaps. F1.1 and F1.2, this brightness might be conveyed in units of reflectance, radiance, or digital number (DN); the NDVI is intended to give nearly equivalent values across platforms that use these wavelengths. The general form of this equation is called a “normalized difference”—the numerator is the “difference” and the denominator “normalizes” the value. Outputs for NDVI vary between −1 and 1. High amounts of green vegetation have values around 0.8–0.9. Absence of green leaves gives values near 0, and water gives values near −1.</p>
|
||||
<p>To compute the NDVI, we will introduce Earth Engine’s implementation of band arithmetic. Cloud-based band arithmetic is one of the most powerful aspects of Earth Engine, because the platform’s computers are optimized for this type of heavy processing. Arithmetic on bands can be done even at planetary scale very quickly—an idea that was out of reach before the advent of cloud-based remote sensing. Earth Engine automatically partitions calculations across a large number of computers as needed, and assembles the answer for display.</p>
|
||||
<p>As an example, let’s examine an image of San Francisco (Fig. F2.0.3).</p>
|
||||
@@ -435,10 +435,11 @@ Chapter Information
|
||||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(sfoImage<span class="op">,</span> { </span>
|
||||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">bands</span><span class="op">:</span> [<span class="st">'B8'</span><span class="op">,</span> <span class="st">'B4'</span><span class="op">,</span> <span class="st">'B3'</span>]<span class="op">,</span> </span>
|
||||
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> </span>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">2000</span>}<span class="op">,</span> <span class="st">'False color'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">2000</span>}<span class="op">,</span> <span class="st">'False color'</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image46.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image46.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.3 False color Sentinel-2 imagery of San Francisco and surroundings</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -460,18 +461,19 @@ Chapter Information
|
||||
<span id="cb2-15"><a href="#cb2-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb2-16"><a href="#cb2-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb2-17"><a href="#cb2-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> vegPalette </span>
|
||||
<span id="cb2-18"><a href="#cb2-18" aria-hidden="true" tabindex="-1"></a>}<span class="op">,</span> <span class="st">'NDVI Manual'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb2-18"><a href="#cb2-18" aria-hidden="true" tabindex="-1"></a>}<span class="op">,</span> <span class="st">'NDVI Manual'</span>)<span class="op">;</span></span>
|
||||
<span id="cb2-19"><a href="#cb2-19" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Examine the resulting index, using the Inspector to pick out pixel values in areas of vegetation and non-vegetation if desired.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image50.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image50.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.4 NDVI calculated using Sentinel-2. Remember that outputs for NDVI vary between −1 and 1. High amounts of green vegetation have values around 0.8–0.9. Absence of green leaves gives values near 0, and water gives values near −1.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Using these simple arithmetic tools, you can build almost any index, or develop and visualize your own. Earth Engine allows you to quickly and easily calculate and display the index across a large area.</p>
|
||||
</section>
|
||||
<section id="single-operation-computation-of-normalized-difference-for-ndvi" class="level4" data-number="2.1.1.2">
|
||||
<h4 data-number="2.1.1.2" class="anchored" data-anchor-id="single-operation-computation-of-normalized-difference-for-ndvi"><span class="header-section-number">2.1.1.2</span> Single-Operation Computation of Normalized Difference for NDVI</h4>
|
||||
<section id="single-operation-computation-of-normalized-difference-for-ndvi" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="single-operation-computation-of-normalized-difference-for-ndvi">Single-Operation Computation of Normalized Difference for NDVI</h4>
|
||||
<p>Normalized differences like NDVI are so common in remote sensing that Earth Engine provides the ability to do that particular sequence of subtraction, addition, and division in a single step, using the normalizedDifference method. This method takes an input image, along with bands you specify, and creates a normalized difference of those two bands. The NDVI computation previously created with band arithmetic can be replaced with one line of code:</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Now use the built-in normalizedDifference function to achieve the same outcome. </span></span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ndviND <span class="op">=</span> sfoImage<span class="op">.</span><span class="fu">normalizedDifference</span>([<span class="st">'B8'</span><span class="op">,</span> <span class="st">'B4'</span>])<span class="op">;</span> </span>
|
||||
@@ -479,16 +481,17 @@ Chapter Information
|
||||
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> vegPalette </span>
|
||||
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a>}<span class="op">,</span> <span class="st">'NDVI normalizedDiff'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a>}<span class="op">,</span> <span class="st">'NDVI normalizedDiff'</span>)<span class="op">;</span></span>
|
||||
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Note that the order in which you provide the two bands to normalizedDifference is important. We use B8, the near-infrared band, as the first parameter, and the red band B4 as the second. If your two computations of NDVI do not look identical when drawn to the screen, check to make sure that the order you have for the NIR and red bands is correct.</p>
|
||||
</section>
|
||||
<section id="using-normalized-difference-for-ndwi" class="level4" data-number="2.1.1.3">
|
||||
<h4 data-number="2.1.1.3" class="anchored" data-anchor-id="using-normalized-difference-for-ndwi"><span class="header-section-number">2.1.1.3</span> Using Normalized Difference for NDWI</h4>
|
||||
<section id="using-normalized-difference-for-ndwi" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="using-normalized-difference-for-ndwi">Using Normalized Difference for NDWI</h4>
|
||||
<p>As mentioned, the normalized difference approach is used for many different indices. Let’s apply the same normalizedDifference method to another index.</p>
|
||||
<p>The Normalized Difference Water Index (NDWI) was developed by Gao (1996) as an index of vegetation water content. The index is sensitive to changes in the liquid content of vegetation canopies. This means that the index can be used, for example, to detect vegetation experiencing drought conditions or differentiate crop irrigation levels. In dry areas, crops that are irrigated can be differentiated from natural vegetation. It is also sometimes called the Normalized Difference Moisture Index (NDMI). NDWI is formulated as follows:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image2.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image2.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">(F2.0.2)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -501,11 +504,12 @@ Chapter Information
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="fl">0.5</span><span class="op">,</span> </span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> waterPalette </span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a>}<span class="op">,</span> <span class="st">'NDWI'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a>}<span class="op">,</span> <span class="st">'NDWI'</span>)<span class="op">;</span></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Examine the areas of the map that NDVI identified as having a lot of vegetation. Notice which are more blue. This is vegetation that has higher water content.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image40.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image40.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.5 NDWI displayed for Sentinel-2 over San Francisco</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -524,11 +528,11 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="thresholding-masking-and-remapping-images" class="level3" data-number="2.1.2">
|
||||
<h3 data-number="2.1.2" class="anchored" data-anchor-id="thresholding-masking-and-remapping-images"><span class="header-section-number">2.1.2</span> Thresholding, Masking, and Remapping Images</h3>
|
||||
<section id="thresholding-masking-and-remapping-images" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="thresholding-masking-and-remapping-images">Thresholding, Masking, and Remapping Images</h3>
|
||||
<p>The previous section in this chapter discussed how to use band arithmetic to manipulate images. Those methods created new continuous values by combining bands within an image. This section uses logical operators to categorize band or index values to create a categorized image.</p>
|
||||
<section id="implementing-a-threshold" class="level4" data-number="2.1.2.1">
|
||||
<h4 data-number="2.1.2.1" class="anchored" data-anchor-id="implementing-a-threshold"><span class="header-section-number">2.1.2.1</span> Implementing a Threshold</h4>
|
||||
<section id="implementing-a-threshold" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="implementing-a-threshold">Implementing a Threshold</h4>
|
||||
<p>Implementing a threshold uses a number (the threshold value) and logical operators to help us partition the variability of images into categories. For example, recall our map of NDVI. High amounts of vegetation have NDVI values near 1 and non-vegetated areas are near 0. If we want to see what areas of the map have vegetation, we can use a threshold to generalize the NDVI value in each pixel as being either “no vegetation” or “vegetation”. That is a substantial simplification, to be sure, but can help us to better comprehend the rich variation on the Earth’s surface. This type of categorization may be useful if, for example, we want to look at the proportion of a city that is vegetated. Let’s create a Sentinel-2 map of NDVI near Seattle, Washington, USA. Enter the code below in a new script.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Create an NDVI image using Sentinel 2. </span></span>
|
||||
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> seaPoint <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Point</span>(<span class="op">-</span><span class="fl">122.2040</span><span class="op">,</span> <span class="fl">47.6221</span>)<span class="op">;</span> </span>
|
||||
@@ -547,10 +551,11 @@ Note
|
||||
<span id="cb5-15"><a href="#cb5-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb5-16"><a href="#cb5-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb5-17"><a href="#cb5-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> vegPalette </span>
|
||||
<span id="cb5-18"><a href="#cb5-18" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'NDVI Seattle'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb5-18"><a href="#cb5-18" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'NDVI Seattle'</span>)<span class="op">;</span></span>
|
||||
<span id="cb5-19"><a href="#cb5-19" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image30.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image30.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.6 NDVI image of Sentinel-2 imagery over Seattle, Washington, USA</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -565,21 +570,22 @@ Note
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> </span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> [<span class="st">'white'</span><span class="op">,</span> <span class="st">'green'</span>] </span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Non-forest vs. Forest'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The gt method is from the family of Boolean operators—that is, gt is a function that performs a test in each pixel and returns the value 1 if the test evaluates to true, and 0 otherwise. Here, for every pixel in the image, it tests whether the NDVI value is greater than 0.5. When this condition is met, the layer seaVeg gets the value 1. When the condition is false, it receives the value 0.</p>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Non-forest vs. Forest'</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The gt method is from the family of Boolean operators — that is, gt is a function that performs a test in each pixel and returns the value 1 if the test evaluates to true, and 0 otherwise. Here, for every pixel in the image, it tests whether the NDVI value is greater than 0.5. When this condition is met, the layer seaVeg gets the value 1. When the condition is false, it receives the value 0.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image47.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image47.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.7 Thresholded forest and non-forest image based on NDVI for Seattle, Washington, USA</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Use the Inspector tool to explore this new layer. If you click on a green location, that NDVI should be greater than 0.5. If you click on a white pixel, the NDVI value should be equal to or less than 0.5.</p>
|
||||
<p>Other operators in this Boolean family include less than (lt), less than or equal to (lte), equal to (eq), not equal to (neq), and greater than or equal to (gte) and more.</p>
|
||||
</section>
|
||||
<section id="building-complex-categorizations-with-.where" class="level4" data-number="2.1.2.2">
|
||||
<h4 data-number="2.1.2.2" class="anchored" data-anchor-id="building-complex-categorizations-with-.where"><span class="header-section-number">2.1.2.2</span> Building Complex Categorizations with .where</h4>
|
||||
<p>A binary map classifying NDVI is very useful. However, there are situations where you may want to split your image into more than two bins. Earth Engine provides a tool, the where method, that conditionally evaluates to true or false within each pixel depending on the outcome of a test. This is analogous to an if statement seen commonly in other languages. However, to perform this logic when programming for Earth Engine, we avoid using the JavaScript if statement. Importantly, JavaScript if commands are not calculated on Google’s servers, and can create serious problems when running your code—in effect, the servers try to ship all of the information to be executed to your own computer’s browser, which is very underequipped for such enormous tasks. Instead, we use the where clause for conditional logic.</p>
|
||||
<p>Suppose instead of just splitting the forested areas from the non-forested areas in our NDVI, we want to split the image into likely water, non-forested, and forested areas. We can use where and thresholds of -0.1 and 0.5. We will start by creating an image using ee.Image. We then clip the new image so that it covers the same area as our seaNDVI layer.</p>
|
||||
<section id="building-complex-categorizations-with-.where" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="building-complex-categorizations-with-.where">Building Complex Categorizations with .where</h4>
|
||||
<p>A binary map classifying NDVI is very useful. However, there are situations where you may want to split your image into more than two bins. Earth Engine provides a tool, the where method, that conditionally evaluates to true or false within each pixel depending on the outcome of a test. This is analogous to an if statement seen commonly in other languages. However, to perform this logic when programming for Earth Engine, we avoid using the JavaScript if statement. Importantly, JavaScript if commands are not calculated on Google’s servers, and can create serious problems when running your code — in effect, the servers try to ship all of the information to be executed to your own computer’s browser, which is very underequipped for such enormous tasks. Instead, we use the where clause for conditional logic.</p>
|
||||
<p>Suppose instead of just splitting the forested areas from the non-forested areas in our NDVI, we want to split the image into likely water, non-forested and forested areas. We can use where and thresholds of -0.1 and 0.5. We will start by creating an image using ee.Image. We then clip the new image so that it covers the same area as our seaNDVI layer.</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Implement .where. </span></span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a starting image with all values = 1. </span></span>
|
||||
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> seaWhere <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>(<span class="dv">1</span>) <span class="co">// Use clip to constrain the size of the new image. .clip(seaNDVI.geometry()); </span></span>
|
||||
@@ -596,32 +602,35 @@ Note
|
||||
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> </span>
|
||||
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">2</span><span class="op">,</span> </span>
|
||||
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> [<span class="st">'blue'</span><span class="op">,</span> <span class="st">'white'</span><span class="op">,</span> <span class="st">'green'</span>] </span>
|
||||
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Water, Non-forest, Forest'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Water, Non-forest, Forest'</span>)<span class="op">;</span></span>
|
||||
<span id="cb7-18"><a href="#cb7-18" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>There are a few interesting things to note about this code that you may not have seen before. First, we’re not defining a new variable for each where call. As a result, we can perform many where calls without creating a new variable each time and needing to keep track of them. Second, when we created the starting image, we set the value to 1. This means that we could easily set the bottom and top values with one where clause each. Finally, while we did not do it here, we can combine multiple where clauses using and and or. For example, we could identify pixels with an intermediate level of NDVI using seaNDVI.gte(-0.1).and(seaNDVI.lt(0.5)).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image37.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image37.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.8 Thresholded water, forest, and non-forest image based on NDVI for Seattle, Washington, USA.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
<section id="masking-specific-values-in-an-image" class="level4" data-number="2.1.2.3">
|
||||
<h4 data-number="2.1.2.3" class="anchored" data-anchor-id="masking-specific-values-in-an-image"><span class="header-section-number">2.1.2.3</span> Masking Specific Values in an Image</h4>
|
||||
<p>Masking an image is a technique that removes specific areas of an image—those covered by the mask—from being displayed or analyzed. Earth Engine allows you to both view the current mask and update the mask.</p>
|
||||
<section id="masking-specific-values-in-an-image" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="masking-specific-values-in-an-image">Masking Specific Values in an Image</h4>
|
||||
<p>Masking an image is a technique that removes specific areas of an image — those covered by the mask — from being displayed or analyzed. Earth Engine allows you to both view the current mask and update the mask.</p>
|
||||
<div class="sourceCode" id="cb8"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Implement masking. </span></span>
|
||||
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a><span class="co">// View the seaVeg layer's current mask. </span></span>
|
||||
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">centerObject</span>(seaPoint<span class="op">,</span> <span class="dv">9</span>)<span class="op">;</span> </span>
|
||||
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(seaVeg<span class="op">.</span><span class="fu">mask</span>()<span class="op">,</span> {}<span class="op">,</span> <span class="st">'seaVeg Mask'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(seaVeg<span class="op">.</span><span class="fu">mask</span>()<span class="op">,</span> {}<span class="op">,</span> <span class="st">'seaVeg Mask'</span>)<span class="op">;</span></span>
|
||||
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image23.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image23.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.9 The existing mask for the seaVeg layer we created previously</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>You can use the Inspector to see that the black area is masked and the white area has a constant value of 1. This means that data values are mapped and available for analysis within the white area only.</p>
|
||||
<p>Now suppose we only want to display and conduct analyses in the forested areas. Let’s mask out the non-forested areas from our image. First, we create a binary mask using the equals (eq) method.</p>
|
||||
<div class="sourceCode" id="cb9"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a binary mask of non-forest. </span></span>
|
||||
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> vegMask <span class="op">=</span> seaVeg<span class="op">.</span><span class="fu">eq</span>(<span class="dv">1</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> vegMask <span class="op">=</span> seaVeg<span class="op">.</span><span class="fu">eq</span>(<span class="dv">1</span>)<span class="op">;</span></span>
|
||||
<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>In making a mask, you set the values you want to see and analyze to be a number greater than 0. The idea is to set unwanted values to get the value of 0. Pixels that had 0 values become masked out (in practice, they do not appear on the screen at all) once we use the updateMask method to add these values to the existing mask.</p>
|
||||
<div class="sourceCode" id="cb10"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Update the seaVeg mask with the non-forest mask. </span></span>
|
||||
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> maskedVeg <span class="op">=</span> seaVeg<span class="op">.</span><span class="fu">updateMask</span>(vegMask)<span class="op">;</span> </span>
|
||||
@@ -632,26 +641,28 @@ Note
|
||||
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> </span>
|
||||
<span id="cb10-8"><a href="#cb10-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb10-9"><a href="#cb10-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> [<span class="st">'green'</span>] </span>
|
||||
<span id="cb10-10"><a href="#cb10-10" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Masked Forest Layer'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb10-10"><a href="#cb10-10" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Masked Forest Layer'</span>)<span class="op">;</span></span>
|
||||
<span id="cb10-11"><a href="#cb10-11" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Turn off all of the other layers. You can see how the maskedVeg layer now has masked out all non-forested areas.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image26.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image26.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.10 An updated mask now displays only the forested areas. Non-forested areas are masked out and transparent.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Map the updated mask for the layer and you can see why this is.</p>
|
||||
<div class="sourceCode" id="cb11"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Map the updated mask </span></span>
|
||||
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(maskedVeg<span class="op">.</span><span class="fu">mask</span>()<span class="op">,</span> {}<span class="op">,</span> <span class="st">'maskedVeg Mask'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(maskedVeg<span class="op">.</span><span class="fu">mask</span>()<span class="op">,</span> {}<span class="op">,</span> <span class="st">'maskedVeg Mask'</span>)<span class="op">;</span></span>
|
||||
<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image33.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image33.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.11 The updated mask. Areas of non-forest are now masked out as well (black areas of the image).</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
<section id="remapping-values-in-an-image" class="level4" data-number="2.1.2.4">
|
||||
<h4 data-number="2.1.2.4" class="anchored" data-anchor-id="remapping-values-in-an-image"><span class="header-section-number">2.1.2.4</span> Remapping Values in an Image</h4>
|
||||
<section id="remapping-values-in-an-image" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="remapping-values-in-an-image">Remapping Values in an Image</h4>
|
||||
<p>Remapping takes specific values in an image and assigns them a different value. This is particularly useful for categorical datasets, including those you read about in Chap. F1.2 and those we have created earlier in this chapter.</p>
|
||||
<p>Let’s use the remap method to change the values for our seaWhere layer. Note that since we’re changing the middle value to be the largest, we’ll need to adjust our palette as well.</p>
|
||||
<div class="sourceCode" id="cb12"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Implement remapping. </span></span>
|
||||
@@ -663,11 +674,12 @@ Note
|
||||
<span id="cb12-7"><a href="#cb12-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">9</span><span class="op">,</span> </span>
|
||||
<span id="cb12-8"><a href="#cb12-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">11</span><span class="op">,</span> </span>
|
||||
<span id="cb12-9"><a href="#cb12-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">palette</span><span class="op">:</span> [<span class="st">'blue'</span><span class="op">,</span> <span class="st">'green'</span><span class="op">,</span> <span class="st">'white'</span>] </span>
|
||||
<span id="cb12-10"><a href="#cb12-10" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Remapped Values'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb12-10"><a href="#cb12-10" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span> <span class="st">'Remapped Values'</span>)<span class="op">;</span></span>
|
||||
<span id="cb12-11"><a href="#cb12-11" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Use the inspector to compare values between our original seaWhere (displayed as Water, Non-Forest, Forest) and the seaRemap, marked as “Remapped Values.” Click on a forested area and you should see that the Remapped Values should be 10, instead of 2 (Fig. F2.0.12).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image28.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image28.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.0.12 For forested areas, the remapped layer has a value of 10, compared with the original layer, which has a value of 2. You may have more layers in your Inspector.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -707,8 +719,8 @@ Note
|
||||
<p>Souza Jr CM, Siqueira JV, Sales MH, et al (2013) Ten-year Landsat classification of deforestation and forest degradation in the Brazilian Amazon. Remote Sens 5:5493–5513. https://doi.org/10.3390/rs5115493</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="interpreting-an-image-classification" class="level2" data-number="2.2">
|
||||
<h2 data-number="2.2" class="anchored" data-anchor-id="interpreting-an-image-classification"><span class="header-section-number">2.2</span> Interpreting an Image: Classification</h2>
|
||||
<section id="interpreting-an-image-classification" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="interpreting-an-image-classification">Interpreting an Image: Classification</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -725,7 +737,7 @@ Chapter Information
|
||||
</section>
|
||||
<section id="overview-1" class="level4 unlisted unnumbered">
|
||||
<h4 class="unlisted unnumbered anchored" data-anchor-id="overview-1">Overview</h4>
|
||||
<p>Image classification is a fundamental goal of remote sensing. It takes the user from viewing an image to labeling its contents. This chapter introduces readers to the concept of classification and walks users through the many options for image classification in Earth Engine. You will explore the processes of training data collection, classifier selection, classifier training, and image classification.</p>
|
||||
<p>Image classification is a fundamental goal of remote sensing. It takes the user from viewing an image to labeling its contents. This chapter introduces readers to the concept of classification and walks users through the many options for image classification in Earth Engine. You will explore the processes of training data collection, classifier selection, classifier training and image classification.</p>
|
||||
</section>
|
||||
<section id="learning-outcomes-1" class="level4 unlisted unnumbered">
|
||||
<h4 class="unlisted unnumbered anchored" data-anchor-id="learning-outcomes-1">Learning Outcomes</h4>
|
||||
@@ -740,7 +752,7 @@ Chapter Information
|
||||
<section id="assumes-you-know-how-to-1" class="level4 unlisted unnumbered">
|
||||
<h4 class="unlisted unnumbered anchored" data-anchor-id="assumes-you-know-how-to-1">Assumes you know how to:</h4>
|
||||
<ul>
|
||||
<li>Import images and image collections, filter, and visualize (Part F1).</li>
|
||||
<li>Import images and image collections, filter and visualize (Part F1).</li>
|
||||
<li>Understand bands and how to select them (Chap. F1.2, Chap. F2.0).</li>
|
||||
</ul>
|
||||
</section>
|
||||
@@ -748,19 +760,19 @@ Chapter Information
|
||||
</div>
|
||||
<section id="introduction-1" class="level3 unlisted unnumbered">
|
||||
<h3 class="unlisted unnumbered anchored" data-anchor-id="introduction-1">Introduction</h3>
|
||||
<p>Classification is addressed in a broad range of fields, including mathematics, statistics, data mining, machine learning, and more. For a deeper treatment of classification, interested readers may see some of the following suggestions: Witten et al. (2011), Hastie et al. (2009), Goodfellow et al. (2016), Gareth et al. (2013), Géron (2019), Müller et al. (2016), or Witten et al. (2005). Unlike regression, which predicts continuous variables, classification predicts categorical, or discrete, variables—variables with a finite number of categories (e.g., age range).</p>
|
||||
<p>In remote sensing, image classification is an attempt to categorize all pixels in an image into a finite number of labeled land cover and/or land use classes. The resulting classified image is a simplified thematic map derived from the original image (Fig. F2.1.1). Land cover and land use information is essential for many environmental and socioeconomic applications, including natural resource management, urban planning, biodiversity conservation, agricultural monitoring, and carbon accounting.</p>
|
||||
<p>Classification is addressed in a broad range of fields, including mathematics, statistics, data mining, machine learning and more. For a deeper treatment of classification, interested readers may see some of the following suggestions: Witten et al. (2011), Hastie et al. (2009), Goodfellow et al. (2016), Gareth et al. (2013), Géron (2019), Müller et al. (2016), or Witten et al. (2005). Unlike regression, which predicts continuous variables, classification predicts categorical, or discrete, variables — those with a finite number of categories (e.g., age range).</p>
|
||||
<p>In remote sensing, image classification is an attempt to categorize all pixels in an image into a finite number of labeled land cover and/or land use classes. The resulting classified image is a simplified thematic map derived from the original image (Fig. F2.1.1). Land cover and land use information is essential for many environmental and socioeconomic applications, including natural resource management, urban planning, biodiversity conservation, agricultural monitoring and carbon accounting.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image48.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image48.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.1 Image classification concept</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Image classification techniques for generating land cover and land use information have been in use since the 1980s (Li et al. 2014). Here, we will cover the concepts of pixel-based supervised and unsupervised classifications, testing out different classifiers. Chapter F3.3 covers the concept and application of object-based classification.</p>
|
||||
<p>It is important to define land use and land cover. Land cover relates to the physical characteristics of the surface: simply put, it documents whether an area of the Earth’s surface is covered by forests, water, impervious surfaces, etc. Land use refers to how this land is being used by people. For example, herbaceous vegetation is considered a land cover but can indicate different land uses: the grass in a pasture is an agricultural land use, whereas the grass in an urban area can be classified as a park.</p>
|
||||
</section>
|
||||
<section id="supervised-classification" class="level3" data-number="2.2.1">
|
||||
<h3 data-number="2.2.1" class="anchored" data-anchor-id="supervised-classification"><span class="header-section-number">2.2.1</span> Supervised Classification</h3>
|
||||
<section id="supervised-classification" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="supervised-classification">Supervised Classification</h3>
|
||||
<p>If you have not already done so, be sure to add the book’s code repository to the Code Editor by entering <a href="https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo%3Dprojects/gee-edu/book&sa=D&source=editors&ust=1671458829866098&usg=AOvVaw16x5swm9HlorS5Mbw7E42X"></a><a href="https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo%3Dprojects/gee-edu/book&sa=D&source=editors&ust=1671458829866485&usg=AOvVaw0-N-JCWWgnM493BKa7Ichm">https://code.earthengine.google.com/?accept_repo=projects/gee-edu/book</a> into your browser. The book’s scripts will then be available in the script manager panel. If you have trouble finding the repo, you can visit <a href="https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit%23slide%3Did.g18a7b4b055d_0_624&sa=D&source=editors&ust=1671458829866823&usg=AOvVaw0ytMyRvutssBcVr2GdcBHA">this link</a> for help.</p>
|
||||
<p>Supervised classification uses a training dataset with known labels and representing the spectral characteristics of each land cover class of interest to “supervise” the classification. The overall approach of a supervised classification in Earth Engine is summarized as follows:</p>
|
||||
<ol type="1">
|
||||
@@ -789,10 +801,11 @@ Chapter Information
|
||||
<span id="cb13-17"><a href="#cb13-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">7000</span><span class="op">,</span> </span>
|
||||
<span id="cb13-18"><a href="#cb13-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">12000</span> </span>
|
||||
<span id="cb13-19"><a href="#cb13-19" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span> </span>
|
||||
<span id="cb13-20"><a href="#cb13-20" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(landsat<span class="op">,</span> visParams<span class="op">,</span> <span class="st">'Landsat 8 image'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb13-20"><a href="#cb13-20" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(landsat<span class="op">,</span> visParams<span class="op">,</span> <span class="st">'Landsat 8 image'</span>)<span class="op">;</span></span>
|
||||
<span id="cb13-21"><a href="#cb13-21" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image44.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image44.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.2 Landsat image</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -806,44 +819,44 @@ Chapter Information
|
||||
<p>In the Geometry Tools, click on the marker option (Fig. F2.1.3). This will create a point geometry which will show up as an import named “geometry”. Click on the gear icon to configure this import.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image22.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image22.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.3 Creating a new layer in the Geometry Imports</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>We will start by collecting forest points, so name the import forest. Import it as a FeatureCollection, and then click + Property. Name the new property “class” and give it a value of 0 (Fig. F2.1.4). We can also choose a color to represent this class. For a forest class, it is natural to choose a green color. You can choose the color you prefer by clicking on it, or, for more control, you can use a hexadecimal value.</p>
|
||||
<p>Hexadecimal values are used throughout the digital world to represent specific colors across computers and operating systems. They are specified by six values arranged in three pairs, with one pair each for the red, green, and blue brightness values. If you’re unfamiliar with hexadecimal values, imagine for a moment that colors were specified in pairs of base 10 numbers instead of pairs of base 16. In that case, a bright pure red value would be “990000”; a bright pure green value would be “009900”; and a bright pure blue value would be “000099”. A value like “501263” would be a mixture of the three colors, not especially bright, having roughly equal amounts of blue and red, and much less green: a color that would be a shade of purple. To create numbers in the hexadecimal system, which might feel entirely natural if humans had evolved to have 16 fingers, sixteen “digits” are needed: a base 16 counter goes 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F, then 10, 11, and so on. Given that counting framework, the number “FF” is like “99” in base 10: the largest two-digit number. The hexadecimal color used for coloring the letters of the word FeatureCollection in this book, a color with roughly equal amounts of blue and red, and much less green, is “7F1FA2”</p>
|
||||
<p>Returning to the coloring of the forest points, the hexadecimal value “589400” is a little bit of red, about twice as much green, and no blue: the deep green seen in Figure F2.1.4. Enter that value, with or without the “#” in front, and click OK after finishing the configuration.</p>
|
||||
<p>Hexadecimal values are used throughout the digital world to represent specific colors across computers and operating systems. They are specified by six values arranged in three pairs, with one pair each for the red, green and blue brightness values. If you’re unfamiliar with hexadecimal values, imagine for a moment that colors were specified in pairs of base 10 numbers instead of pairs of base 16. In that case, a bright pure red value would be “990000”; a bright pure green value would be “009900”; and a bright pure blue value would be “000099”. A value like “501263” would be a mixture of the three colors, not especially bright, having roughly equal amounts of blue and red, and much less green: a color that would be a shade of purple. To create numbers in the hexadecimal system, which might feel entirely natural if humans had evolved to have 16 fingers, sixteen “digits” are needed: a base 16 counter goes 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F, then 10, 11, and so on. Given that counting framework, the number “FF” is like “99” in base 10: the largest two-digit number. The hexadecimal color used for coloring the letters of the word FeatureCollection in this book, a color with roughly equal amounts of blue and red and much less green, is “7F1FA2”</p>
|
||||
<p>Returning to the coloring of the forest points, the hexadecimal value “589400” is a little bit of red, about twice as much green and no blue: the deep green seen in Figure F2.1.4. Enter that value, with or without the “#” in front, and click OK after finishing the configuration.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image36.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image36.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.4 Edit geometry layer properties</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Now, in the Geometry Imports, we will see that the import has been renamed forest. Click on it to activate the drawing mode (Fig. F2.1.5) in order to start collecting forest points.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image29.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image29.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.5 Activate forest layer to start collection</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Now, start collecting points over forested areas (Fig. F2.1.6). Zoom in and out as needed. You can use the satellite basemap to assist you, but the basis of your collection should be the Landsat image. Remember that the more points you collect, the more the classifier will learn from the information you provide. For now, let’s set a goal to collect 25 points per class. Click Exit next to Point drawing (Fig. F2.1.5) when finished.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image38.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image38.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.6 Forest points</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Repeat the same process for the other classes by creating new layers (Fig. F2.1.7). Don’t forget to import using the FeatureCollection option as mentioned above. For the developed class, collect points over urban areas. For the water class, collect points over the Ligurian Sea, and also look for other bodies of water, like rivers. For the herbaceous class, collect points over agricultural fields. Remember to set the “class” property for each class to its corresponding code (see Table 2.1.1) and click Exit once you finalize collecting points for each class as mentioned above. We will be using the following hexadecimal colors for the other classes: #FF0000 for developed, #1A11FF for water, and #D0741E for herbaceous.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image41.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image41.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.7 New layer option in Geometry Imports</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>You should now have four FeatureCollection imports named forest, developed, water, and herbaceous (Fig. F2.1.8).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image42.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image42.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.8 Example of training points</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -861,11 +874,12 @@ Note
|
||||
</div>
|
||||
</div>
|
||||
<p>If you wish to have the exact same results demonstrated in this chapter from now on, continue beginning with this Code Checkpoint. If you use the points collected yourself, the results may vary from this point forward.</p>
|
||||
<p>The next step is to combine all the training feature collections into one. Copy and paste the code below to combine them into one FeatureCollection called trainingFeatures. Here, we use the flatten method to avoid having a collection of feature collections—we want individual features within our FeatureCollection.</p>
|
||||
<p>The next step is to combine all the training feature collections into one. Copy and paste the code below to combine them into one FeatureCollection called trainingFeatures. Here, we use the flatten method to avoid having a collection of feature collections — we want individual features within our FeatureCollection.</p>
|
||||
<div class="sourceCode" id="cb14"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Combine training feature collections. </span></span>
|
||||
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> trainingFeatures <span class="op">=</span> ee<span class="op">.</span><span class="fu">FeatureCollection</span>([ </span>
|
||||
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a> forest<span class="op">,</span> developed<span class="op">,</span> water<span class="op">,</span> herbaceous </span>
|
||||
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a>])<span class="op">.</span><span class="fu">flatten</span>()<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true" tabindex="-1"></a>])<span class="op">.</span><span class="fu">flatten</span>()<span class="op">;</span></span>
|
||||
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Note: Alternatively, you could use an existing set of reference data. For example, the European Space Agency (ESA) WorldCover dataset is a global map of land use and land cover derived from ESA’s Sentinel-2 imagery at 10 m resolution. With existing datasets, we can randomly place points on pixels classified as the classes of interest (if you are curious, you can explore the Earth Engine documentation to learn about the ee.Image.stratifiedSample and the ee.FeatureCollection.randomPoints methods). The drawback is that these global datasets will not always contain the specific classes of interest for your region, or may not be entirely accurate at the local scale. Another option is to use samples that were collected in the field (e.g., GPS points). In Chap. F5.0, you will see how to upload your own data as Earth Engine assets.</p>
|
||||
<p>In the combined FeatureCollection, each Feature point should have a property called “class”. The class values are consecutive integers from 0 to 3 (you could verify that this is true by printing trainingFeatures and checking the properties of the features).</p>
|
||||
<p>Now that we have our training points, copy and paste the code below to extract the band information for each class at each point location. First, we define the prediction bands to extract different spectral and thermal information from different bands for each class. Then, we use the sampleRegions method to sample the information from the Landsat image at each point location. This method requires information about the FeatureCollection (our reference points), the property to extract (“class”), and the pixel scale (in meters).</p>
|
||||
@@ -878,18 +892,19 @@ Note
|
||||
<span id="cb15-7"><a href="#cb15-7" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">sampleRegions</span>({ </span>
|
||||
<span id="cb15-8"><a href="#cb15-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> trainingFeatures<span class="op">,</span> </span>
|
||||
<span id="cb15-9"><a href="#cb15-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">properties</span><span class="op">:</span> [<span class="st">'class'</span>]<span class="op">,</span> </span>
|
||||
<span id="cb15-10"><a href="#cb15-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> <span class="dv">30</span> })<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb15-10"><a href="#cb15-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> <span class="dv">30</span> })<span class="op">;</span></span>
|
||||
<span id="cb15-11"><a href="#cb15-11" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>You can check whether the classifierTraining object extracted the properties of interest by printing it and expanding the first feature. You should see the band and class information (Fig. F2.1.9).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image20.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image20.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.9 Example of extracted band information for one point of class 0 (forest)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Now we can choose a classifier. The choice of classifier is not always obvious, and there are many options from which to pick—you can quickly expand the ee.Classifier object under Docs to get an idea of how many options we have for image classification. Therefore, we will be testing different classifiers and comparing their results. We will start with a Classification and Regression Tree (CART) classifier, a well-known classification algorithm (Fig. F2.1.10) that has been around for decades.</p>
|
||||
<p>Now we can choose a classifier. The choice of classifier is not always obvious, and there are many options from which to pick — you can quickly expand the ee.Classifier object under Docs to get an idea of how many options we have for image classification. Therefore, we will be testing different classifiers and comparing their results. We will start with a Classification and Regression Tree (CART) classifier, a well-known classification algorithm (Fig. F2.1.10) that has been around for decades.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image25.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image25.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.10 Example of a decision tree for satellite image classification. Values and classes are hypothetical.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -901,7 +916,8 @@ Note
|
||||
<span id="cb16-5"><a href="#cb16-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">features</span><span class="op">:</span> classifierTraining<span class="op">,</span> </span>
|
||||
<span id="cb16-6"><a href="#cb16-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">classProperty</span><span class="op">:</span> <span class="st">'class'</span><span class="op">,</span> </span>
|
||||
<span id="cb16-7"><a href="#cb16-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">inputProperties</span><span class="op">:</span> predictionBands </span>
|
||||
<span id="cb16-8"><a href="#cb16-8" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb16-8"><a href="#cb16-8" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb16-9"><a href="#cb16-9" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Essentially, the classifier contains the mathematical rules that link labels to spectral information. If you print the variable classifier and expand its properties, you can confirm the basic characteristics of the object (bands, properties, and classifier being used). If you print classifier.explain, you can find a property called “tree” that contains the decision rules.</p>
|
||||
<p>After training the classifier, copy and paste the code below to classify the Landsat image and add it to the Map.</p>
|
||||
<div class="sourceCode" id="cb17"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Classify the Landsat image. </span></span>
|
||||
@@ -915,26 +931,27 @@ Note
|
||||
<span id="cb17-9"><a href="#cb17-9" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span> </span>
|
||||
<span id="cb17-10"><a href="#cb17-10" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb17-11"><a href="#cb17-11" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the classified image to the map. </span></span>
|
||||
<span id="cb17-12"><a href="#cb17-12" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(classified<span class="op">,</span> classificationVis<span class="op">,</span> <span class="st">'CART classified'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb17-12"><a href="#cb17-12" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(classified<span class="op">,</span> classificationVis<span class="op">,</span> <span class="st">'CART classified'</span>)<span class="op">;</span></span>
|
||||
<span id="cb17-13"><a href="#cb17-13" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Note that, in the visualization parameters, we define a palette parameter which in this case represents colors for each pixel value (0–3, our class codes). We use the same hexadecimal colors used when creating our training points for each class. This way, we can associate a color with a class when visualizing the classified image in the Map.</p>
|
||||
<p>Inspect the result: Activate the Landsat composite layer and the satellite basemap to overlay with the classified images (Fig. F2.1.11). Change the layers’ transparency to inspect some areas. What do you notice? The result might not look very satisfactory in some areas (e.g., confusion between developed and herbaceous classes). Why do you think this is happening? There are a few options to handle misclassification errors:</p>
|
||||
<ul>
|
||||
<li>Collect more training data We can try incorporating more points to have a more representative sample of the classes.</li>
|
||||
<li>Tune the model Classifiers typically have “hyperparameters,” which are set to default values. In the case of classification trees, there are ways to tune the number of leaves in the tree, for example. Tuning models is addressed in Chap. F2.2.</li>
|
||||
<li>Try other classifiers If a classifier’s results are unsatisfying, we can try some of the other classifiers in Earth Engine to see if the result is better or different.</li>
|
||||
<li>Expand the collection location It is good practice to collect points across the entire image and not just focus on one location. Also, look for pixels of the same class that show variability (e.g., for the developed class, building rooftops look different than house rooftops; for the herbaceous class, crop fields show distinctive seasonality/phenology).</li>
|
||||
<li>Add more predictors We can try adding spectral indices to the input variables; this way, we are feeding the classifier new, unique information about each class. For example, there is a good chance that a vegetation index specialized for detecting vegetation health (e.g., NDVI) would improve the developed versus herbaceous classification.</li>
|
||||
<li>Tune the model. Classifiers typically have “hyperparameters,” which are set to default values. In the case of classification trees, there are ways to tune the number of leaves in the tree, for example. Tuning models is addressed in Chap. F2.2.</li>
|
||||
<li>Try other classifiers. If a classifier’s results are unsatisfying, we can try some of the other classifiers in Earth Engine to see if the result is better or different.</li>
|
||||
<li>Expand the collection location. It is good practice to collect points across the entire image and not just focus on one location. Also, look for pixels of the same class that show variability (e.g., for the developed class, building rooftops look different than house rooftops; for the herbaceous class, crop fields show distinctive seasonality/phenology).</li>
|
||||
<li>Add more predictors. We can try adding spectral indices to the input variables; this way, we are feeding the classifier new, unique information about each class. For example, there is a good chance that a vegetation index specialized for detecting vegetation health (e.g., NDVI) would improve the developed versus herbaceous classification.</li>
|
||||
</ul>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image21.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image21.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.11 CART classification</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>For now, we will try another supervised learning classifier that is widely used: Random Forests (RF). The RF algorithm (Breiman 2001, Pal 2005) builds on the concept of decision trees, but adds strategies to make them more powerful. It is called a “forest” because it operates by constructing a multitude of decision trees. As mentioned previously, a decision tree creates the rules which are used to make decisions. A Random Forest will randomly choose features and make observations, build a forest of decision trees, and then use the full set of trees to estimate the class. It is a great choice when you do not have a lot of insight about the training data.</p>
|
||||
<p>For now, we will try another supervised learning classifier that is widely used: Random Forests (RF). The RF algorithm (Breiman 2001, Pal 2005) builds on the concept of decision trees, but adds strategies to make them more powerful. It is called a “forest” because it operates by constructing a multitude of decision trees. As mentioned previously, a decision tree creates the rules which are used to make decisions. A Random Forest will randomly choose features and make observations, build a forest of decision trees and then use the full set of trees to estimate the class. It is a great choice when you do not have a lot of insight about the training data.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image27.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image27.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.12 General concept of Random Forests</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -953,12 +970,13 @@ Note
|
||||
<span id="cb18-12"><a href="#cb18-12" aria-hidden="true" tabindex="-1"></a> RFclassifier)<span class="op">;</span> </span>
|
||||
<span id="cb18-13"><a href="#cb18-13" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb18-14"><a href="#cb18-14" aria-hidden="true" tabindex="-1"></a><span class="co">// Add classified image to the map. </span></span>
|
||||
<span id="cb18-15"><a href="#cb18-15" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(RFclassified<span class="op">,</span> classificationVis<span class="op">,</span> <span class="st">'RF classified'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb18-15"><a href="#cb18-15" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(RFclassified<span class="op">,</span> classificationVis<span class="op">,</span> <span class="st">'RF classified'</span>)<span class="op">;</span></span>
|
||||
<span id="cb18-16"><a href="#cb18-16" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Note that in the ee.Classifier.smileRandomForest documentation (Docs tab), there is a seed (random number) parameter. Setting a seed allows you to exactly replicate your model each time you run it. Any number is acceptable as a seed.</p>
|
||||
<p>Inspect the result (Fig. F2.1.13). How does this classified image differ from the CART one? Is the classifications better or worse? Zoom in and out and change the transparency of layers as needed. In Chap. F2.2, you will see more systematic ways to assess what is better or worse, based on accuracy metrics.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image34.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image34.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.13 Random Forest classified image</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -976,9 +994,9 @@ Note
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="unsupervised-classification" class="level3" data-number="2.2.2">
|
||||
<h3 data-number="2.2.2" class="anchored" data-anchor-id="unsupervised-classification"><span class="header-section-number">2.2.2</span> Unsupervised Classification</h3>
|
||||
<p>In an unsupervised classification, we have the opposite process of supervised classification. Spectral classes are grouped first and then categorized into clusters. Therefore, in Earth Engine, these classifiers are ee.Clusterer objects. They are “self-taught” algorithms that do not use a set of labeled training data (i.e., they are “unsupervised”). You can think of it as performing a task that you have not experienced before, starting by gathering as much information as possible. For example, imagine learning a new language without knowing the basic grammar, learning only by watching a TV series in that language, listening to examples, and finding patterns.</p>
|
||||
<section id="unsupervised-classification" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="unsupervised-classification">Unsupervised Classification</h3>
|
||||
<p>In an unsupervised classification, we have the opposite process of supervised classification. Spectral classes are grouped first and then categorized into clusters. Therefore, in Earth Engine, these classifiers are ee.Clusterer objects. They are “self-taught” algorithms that do not use a set of labeled training data (i.e., they are “unsupervised”). You can think of it as performing a task that you have not experienced before, starting by gathering as much information as possible. For example, imagine learning a new language without knowing the basic grammar, learning only by watching a TV series in that language, listening to examples and finding patterns.</p>
|
||||
<p>Similar to the supervised classification, unsupervised classification in Earth Engine has this workflow:</p>
|
||||
<ol type="1">
|
||||
<li>Assemble features with numeric properties in which to find clusters (training data).</li>
|
||||
@@ -987,7 +1005,7 @@ Note
|
||||
<li>Apply the clusterer to the scene (classification).</li>
|
||||
<li>Label the clusters.</li>
|
||||
</ol>
|
||||
<p>In order to generate training data, we will use the sample method, which randomly takes samples from a region (unlike sampleRegions, which takes samples from predefined locations). We will use the image’s footprint as the region by calling the geometry method. Additionally, we will define the number of pixels (numPixels) to sample—in this case, 1000 pixels—and define a tileScale of 8 to avoid computation errors due to the size of the region. Copy and paste the code below to sample 1000 pixels from the Landsat image. You should add to the same script as before to compare supervised versus unsupervised classification results at the end.</p>
|
||||
<p>In order to generate training data, we will use the sample method, which randomly takes samples from a region (unlike sampleRegions, which takes samples from predefined locations). We will use the image’s footprint as the region by calling the geometry method. Additionally, we will define the number of pixels (numPixels) to sample — in this case, 1000 pixels — and define a tileScale of 8 to avoid computation errors due to the size of the region. Copy and paste the code below to sample 1000 pixels from the Landsat image. You should add to the same script as before to compare supervised versus unsupervised classification results at the end.</p>
|
||||
<div class="sourceCode" id="cb19"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="co">//////////////// Unsupervised classification //////////////// </span></span>
|
||||
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb19-3"><a href="#cb19-3" aria-hidden="true" tabindex="-1"></a><span class="co">// Make the training dataset. </span></span>
|
||||
@@ -996,26 +1014,29 @@ Note
|
||||
<span id="cb19-6"><a href="#cb19-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> <span class="dv">30</span><span class="op">,</span> </span>
|
||||
<span id="cb19-7"><a href="#cb19-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">numPixels</span><span class="op">:</span> <span class="dv">1000</span><span class="op">,</span> </span>
|
||||
<span id="cb19-8"><a href="#cb19-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">tileScale</span><span class="op">:</span> <span class="dv">8</span> </span>
|
||||
<span id="cb19-9"><a href="#cb19-9" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb19-9"><a href="#cb19-9" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb19-10"><a href="#cb19-10" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now we can instantiate a clusterer and train it. As with the supervised algorithms, there are many unsupervised algorithms to choose from. We will use the k-means clustering algorithm, which is a commonly used approach in remote sensing. This algorithm identifies groups of pixels near each other in the spectral space (image x bands) by using an iterative regrouping strategy. We define a number of clusters, k, and then the method randomly distributes that number of seed points into the spectral space. A large sample of pixels is then grouped into its closest seed, and the mean spectral value of this group is calculated. That mean value is akin to a center of mass of the points, and is known as the centroid. Each iteration recalculates the class means and reclassifies pixels with respect to the new means. This process is repeated until the centroids remain relatively stable and only a few pixels change from class to class on subsequent iterations.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image35.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image35.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.14 K-means visual concept</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Copy and paste the code below to request four clusters, the same number as for the supervised classification, in order to directly compare them.</p>
|
||||
<div class="sourceCode" id="cb20"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb20-1"><a href="#cb20-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Instantiate the clusterer and train it. </span></span>
|
||||
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> clusterer <span class="op">=</span> ee<span class="op">.</span><span class="at">Clusterer</span><span class="op">.</span><span class="fu">wekaKMeans</span>(<span class="dv">4</span>)<span class="op">.</span><span class="fu">train</span>(training)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb20-2"><a href="#cb20-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> clusterer <span class="op">=</span> ee<span class="op">.</span><span class="at">Clusterer</span><span class="op">.</span><span class="fu">wekaKMeans</span>(<span class="dv">4</span>)<span class="op">.</span><span class="fu">train</span>(training)<span class="op">;</span></span>
|
||||
<span id="cb20-3"><a href="#cb20-3" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now copy and paste the code below to apply the clusterer to the image and add the resulting classification to the Map (Fig. F2.1.15). Note that we are using a method called randomVisualizer to assign colors for the visualization. We are not associating the unsupervised classes with the color palette we defined earlier in the supervised classification. Instead, we are assigning random colors to the classes, since we do not yet know which of the unsupervised classes best corresponds to each of the named classes (e.g., forest , herbaceous). Note that the colors in Fig. F1.2.15 might not be the same as you see on your Map, since they are assigned randomly.</p>
|
||||
<div class="sourceCode" id="cb21"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb21-1"><a href="#cb21-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Cluster the input using the trained clusterer. </span></span>
|
||||
<span id="cb21-2"><a href="#cb21-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> Kclassified <span class="op">=</span> landsat<span class="op">.</span><span class="fu">cluster</span>(clusterer)<span class="op">;</span> </span>
|
||||
<span id="cb21-3"><a href="#cb21-3" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb21-4"><a href="#cb21-4" aria-hidden="true" tabindex="-1"></a><span class="co">// Display the clusters with random colors. </span></span>
|
||||
<span id="cb21-5"><a href="#cb21-5" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(Kclassified<span class="op">.</span><span class="fu">randomVisualizer</span>()<span class="op">,</span> {}<span class="op">,</span> <span class="st">'K-means classified - random colors'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb21-5"><a href="#cb21-5" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(Kclassified<span class="op">.</span><span class="fu">randomVisualizer</span>()<span class="op">,</span> {}<span class="op">,</span> <span class="st">'K-means classified - random colors'</span>)<span class="op">;</span></span>
|
||||
<span id="cb21-6"><a href="#cb21-6" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image31.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image31.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.1.15 K-means classification</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1052,8 +1073,8 @@ Note
|
||||
<p>Witten IH, Frank E, Hall MA, et al (2005) Practical machine learning tools and techniques. In: Data Mining. pp 4</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="accuracy-assessment-quantifying-classification-quality" class="level2" data-number="2.3">
|
||||
<h2 data-number="2.3" class="anchored" data-anchor-id="accuracy-assessment-quantifying-classification-quality"><span class="header-section-number">2.3</span> Accuracy Assessment: Quantifying Classification Quality</h2>
|
||||
<section id="accuracy-assessment-quantifying-classification-quality" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="accuracy-assessment-quantifying-classification-quality">Accuracy Assessment: Quantifying Classification Quality</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -1097,8 +1118,8 @@ Chapter Information
|
||||
<p>In Chap. F2.1, we asked whether the classification results were satisfactory. In remote sensing, the quantification of the answer to that question is called accuracy assessment. In the classification context, accuracy measurements are often derived from a confusion matrix.</p>
|
||||
<p>In a thorough accuracy assessment, we think carefully about the sampling design, the response design, and the analysis (Olofsson et al. 2014). Fundamental protocols are taken into account to produce scientifically rigorous and transparent estimates of accuracy and area, which requires robust planning and time. In a standard setting, we would calculate the number of samples needed for measuring accuracy (sampling design). Here, we will focus mainly on the last step, analysis, by examining the confusion matrix and learning how to calculate the accuracy metrics. This will be done by partitioning the existing data into training and testing sets.</p>
|
||||
</section>
|
||||
<section id="quantifying-classification-accuracy-through-a-confusion-matrix" class="level3" data-number="2.3.1">
|
||||
<h3 data-number="2.3.1" class="anchored" data-anchor-id="quantifying-classification-accuracy-through-a-confusion-matrix"><span class="header-section-number">2.3.1</span> Quantifying Classification Accuracy Through a Confusion Matrix</h3>
|
||||
<section id="quantifying-classification-accuracy-through-a-confusion-matrix" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="quantifying-classification-accuracy-through-a-confusion-matrix">Quantifying Classification Accuracy Through a Confusion Matrix</h3>
|
||||
<p>If you have not already done so, be sure to add the book’s code repository to the Code Editor by entering <a href="https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo%3Dprojects/gee-edu/book&sa=D&source=editors&ust=1671458829937499&usg=AOvVaw3qqOwSX_A-Pllh6X3X31q4"></a><a href="https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo%3Dprojects/gee-edu/book&sa=D&source=editors&ust=1671458829937976&usg=AOvVaw0WioXIhzue8-WoaX4UtabH">https://code.earthengine.google.com/?accept_repo=projects/gee-edu/book</a> into your browser. The book’s scripts will then be available in the script manager panel. If you have trouble finding the repo, you can visit <a href="https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit%23slide%3Did.g18a7b4b055d_0_624&sa=D&source=editors&ust=1671458829938470&usg=AOvVaw2CH8V3-_qV99EcgMxUAaSO">this link</a> for help.</p>
|
||||
<p>To illustrate some of the basic ideas about classification accuracy, we will revisit the data and location of part of Chap. F2.1, where we tested different classifiers and classified a Landsat image of the area around Milan, Italy. We will name this dataset ‘data’. This variable is a FeatureCollection with features containing the “class” values and spectral information of four land cover / land use classes: forest, developed, water, and herbaceous (see Fig. F2.1.8 and Fig. F2.1.9 for a refresher). We will also define a variable, predictionBands, which is a list of bands that will be used for prediction (classification)—the spectral information in the data variable.</p>
|
||||
<p>Class Values:</p>
|
||||
@@ -1121,7 +1142,8 @@ Chapter Information
|
||||
<span id="cb22-10"><a href="#cb22-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> trainingSet <span class="op">=</span> trainingTesting </span>
|
||||
<span id="cb22-11"><a href="#cb22-11" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">lessThan</span>(<span class="st">'random'</span><span class="op">,</span> <span class="fl">0.8</span>))<span class="op">;</span> </span>
|
||||
<span id="cb22-12"><a href="#cb22-12" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> testingSet <span class="op">=</span> trainingTesting </span>
|
||||
<span id="cb22-13"><a href="#cb22-13" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">greaterThanOrEquals</span>(<span class="st">'random'</span><span class="op">,</span> <span class="fl">0.8</span>))<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb22-13"><a href="#cb22-13" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">greaterThanOrEquals</span>(<span class="st">'random'</span><span class="op">,</span> <span class="fl">0.8</span>))<span class="op">;</span></span>
|
||||
<span id="cb22-14"><a href="#cb22-14" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Note that randomColumn creates pseudorandom numbers in a deterministic way. This makes it possible to generate a reproducible pseudorandom sequence by defining the seed parameter (Earth Engine uses a seed of 0 by default). In other words, given a starting value (i.e., the seed), randomColumn will always provide the same sequence of pseudorandom numbers.</p>
|
||||
<p>Copy and paste the code below to train a Random Forest classifier with 50 decision trees using the trainingSet.</p>
|
||||
<div class="sourceCode" id="cb23"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb23-1"><a href="#cb23-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Train the Random Forest Classifier with the trainingSet. </span></span>
|
||||
@@ -1129,7 +1151,8 @@ Chapter Information
|
||||
<span id="cb23-3"><a href="#cb23-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">features</span><span class="op">:</span> trainingSet<span class="op">,</span> </span>
|
||||
<span id="cb23-4"><a href="#cb23-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">classProperty</span><span class="op">:</span> <span class="st">'class'</span><span class="op">,</span> </span>
|
||||
<span id="cb23-5"><a href="#cb23-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">inputProperties</span><span class="op">:</span> predictionBands </span>
|
||||
<span id="cb23-6"><a href="#cb23-6" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb23-6"><a href="#cb23-6" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb23-7"><a href="#cb23-7" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now, let’s discuss what a confusion matrix is. A confusion matrix describes the quality of a classification by comparing the predicted values to the actual values. A simple example is a confusion matrix for a binary classification into the classes “positive” and “negative,” as shown in Table F2.2.1.</p>
|
||||
<p>Table F2.2.1 Confusion matrix for a binary classification where the classes are “positive” and “negative”</p>
|
||||
<table class="table">
|
||||
@@ -1209,32 +1232,32 @@ Chapter Information
|
||||
</table>
|
||||
<p>In this case, the classifier correctly identified 307 forest pixels, wrongly classified 18 non-forest pixels as forest, correctly identified 661 non-forest pixels, and wrongly classified 14 forest pixels as non-forest. Therefore, the classifier was correct 968 times and wrong 32 times. Let’s calculate the main accuracy metrics for this example.</p>
|
||||
<p>The overall accuracy tells us what proportion of the reference data was classified correctly, and is calculated as the total number of correctly identified pixels divided by the total number of pixels in the sample.</p>
|
||||
<p><img src="F2/image6.png" class="img-fluid"></p>
|
||||
<p>In this case, the overall accuracy is 96.8%, calculated using (<img src="F2/image7.png" class="img-fluid">.</p>
|
||||
<p><img src="../images/F2/image6.png" class="img-fluid"></p>
|
||||
<p>In this case, the overall accuracy is 96.8%, calculated using (<img src="../images/F2/image7.png" class="img-fluid">.</p>
|
||||
<p>Two other important accuracy metrics are the producer’s accuracy and the user’s accuracy, also referred to as the “recall” and the “precision,” respectively. Importantly, these metrics quantify aspects of per-class accuracy.</p>
|
||||
<p>The producer’s accuracy is the accuracy of the map from the point of view of the map maker (the “producer”), and is calculated as the number of correctly identified pixels of a given class divided by the total number of pixels actually in that class. The producer’s accuracy for a given class tells us the proportion of the pixels in that class that were classified correctly.</p>
|
||||
<p><img src="F2/image8.png" class="img-fluid"></p>
|
||||
<p><img src="F2/image9.png" class="img-fluid"></p>
|
||||
<p>In this case, the producer’s accuracy for the forest class is 95.6%, calculated using <img src="F2/image10.png" class="img-fluid">). The producer’s accuracy for the non-forest class is 97.3%, calculated from <img src="F2/image11.png" class="img-fluid">).</p>
|
||||
<p><img src="../images/F2/image8.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F2/image9.png" class="img-fluid"></p>
|
||||
<p>In this case, the producer’s accuracy for the forest class is 95.6%, calculated using <img src="../images/F2/image10.png" class="img-fluid">). The producer’s accuracy for the non-forest class is 97.3%, calculated from <img src="../images/F2/image11.png" class="img-fluid">).</p>
|
||||
<p>The user’s accuracy (also called the “consumer’s accuracy”) is the accuracy of the map from the point of view of a map user, and is calculated as the number of correctly identified pixels of a given class divided by the total number of pixels claimed to be in that class. The user’s accuracy for a given class tells us the proportion of the pixels identified on the map as being in that class that are actually in that class on the ground.</p>
|
||||
<p><img src="F2/image12.png" class="img-fluid"></p>
|
||||
<p><img src="F2/image13.png" class="img-fluid"></p>
|
||||
<p>In this case, the user’s accuracy for the forest class is 94.5%, calculated using <img src="F2/image14.png" class="img-fluid">). The user’s accuracy for the non-forest class is 97.9%, calculated from <img src="F2/image15.png" class="img-fluid">).</p>
|
||||
<p><img src="../images/F2/image12.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F2/image13.png" class="img-fluid"></p>
|
||||
<p>In this case, the user’s accuracy for the forest class is 94.5%, calculated using <img src="../images/F2/image14.png" class="img-fluid">). The user’s accuracy for the non-forest class is 97.9%, calculated from <img src="../images/F2/image15.png" class="img-fluid">).</p>
|
||||
<p>Fig. F2.2.1 helps visualize the rows and columns used to calculate each accuracy.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image43.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image43.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.2.1 Confusion matrix for a binary classification where the classes are “positive” (forest) and “negative” (non-forest), with accuracy metrics</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>It is very common to talk about two types of error when addressing remote-sensing classification accuracy: omission errors and commission errors. Omission errors refer to the reference pixels that were left out of (omitted from) the correct class in the classified map. In a two-class system, an error of omission in one class will be counted as an error of commission in another class. Omission errors are complementary to the producer’s accuracy.</p>
|
||||
<p><img src="F2/image16.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F2/image16.png" class="img-fluid"></p>
|
||||
<p>Commission errors refer to the class pixels that were erroneously classified in the map and are complementary to the user’s accuracy.</p>
|
||||
<p><img src="F2/image17.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F2/image17.png" class="img-fluid"></p>
|
||||
<p>Finally, another commonly used accuracy metric is the kappa coefficient, which evaluates how well the classification performed as compared to random. The value of the kappa coefficient can range from −1 to 1: a negative value indicates that the classification is worse than a random assignment of categories would have been; a value of 0 indicates that the classification is no better or worse than random; and a positive value indicates that the classification is better than random.</p>
|
||||
<p><img src="F2/image18.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F2/image18.png" class="img-fluid"></p>
|
||||
<p>The chance agreement is calculated as the sum of the product of row and column totals for each class, and the observed accuracy is the overall accuracy. Therefore, for our example, the kappa coefficient is 0.927.</p>
|
||||
<p><img src="F2/image19.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F2/image19.png" class="img-fluid"></p>
|
||||
<p>Now, let’s go back to the script. In Earth Engine, there are API calls for these operations. Note that our confusion matrix will be a 4 x 4 table, since we have four different classes.</p>
|
||||
<p>Copy and paste the code below to classify the testingSet and get a confusion matrix using the method errorMatrix. Note that the classifier automatically adds a property called “classification,” which is compared to the “class” property of the reference dataset.</p>
|
||||
<div class="sourceCode" id="cb24"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb24-1"><a href="#cb24-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Now, to test the classification (verify model's accuracy), </span></span>
|
||||
@@ -1242,14 +1265,16 @@ Chapter Information
|
||||
<span id="cb24-3"><a href="#cb24-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> confusionMatrix <span class="op">=</span> testingSet<span class="op">.</span><span class="fu">classify</span>(RFclassifier) </span>
|
||||
<span id="cb24-4"><a href="#cb24-4" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">errorMatrix</span>({ </span>
|
||||
<span id="cb24-5"><a href="#cb24-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">actual</span><span class="op">:</span> <span class="st">'class'</span><span class="op">,</span> </span>
|
||||
<span id="cb24-6"><a href="#cb24-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">predicted</span><span class="op">:</span> <span class="st">'classification'</span> })<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb24-6"><a href="#cb24-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">predicted</span><span class="op">:</span> <span class="st">'classification'</span> })<span class="op">;</span></span>
|
||||
<span id="cb24-7"><a href="#cb24-7" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Copy and paste the code below to print the confusion matrix and accuracy metrics. Expand the confusion matrix object to inspect it. The entries represent the number of pixels. Items on the diagonal represent correct classification. Items off the diagonal are misclassifications, where the class in row i is classified as column j (values from 0 to 3 correspond to our class codes: forest, developed, water, and herbaceous, respectively). Also expand the producer’s accuracy, user’s accuracy (consumer’s accuracy), and kappa coefficient objects to inspect them.</p>
|
||||
<div class="sourceCode" id="cb25"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb25-1"><a href="#cb25-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Print the results. </span></span>
|
||||
<span id="cb25-2"><a href="#cb25-2" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Confusion matrix:'</span><span class="op">,</span> confusionMatrix)<span class="op">;</span> </span>
|
||||
<span id="cb25-3"><a href="#cb25-3" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Overall Accuracy:'</span><span class="op">,</span> confusionMatrix<span class="op">.</span><span class="fu">accuracy</span>())<span class="op">;</span> </span>
|
||||
<span id="cb25-4"><a href="#cb25-4" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Producers Accuracy:'</span><span class="op">,</span> confusionMatrix<span class="op">.</span><span class="fu">producersAccuracy</span>())<span class="op">;</span> </span>
|
||||
<span id="cb25-5"><a href="#cb25-5" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Consumers Accuracy:'</span><span class="op">,</span> confusionMatrix<span class="op">.</span><span class="fu">consumersAccuracy</span>())<span class="op">;</span> </span>
|
||||
<span id="cb25-6"><a href="#cb25-6" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Kappa:'</span><span class="op">,</span> confusionMatrix<span class="op">.</span><span class="fu">kappa</span>())<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb25-6"><a href="#cb25-6" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Kappa:'</span><span class="op">,</span> confusionMatrix<span class="op">.</span><span class="fu">kappa</span>())<span class="op">;</span></span>
|
||||
<span id="cb25-7"><a href="#cb25-7" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>How is the classification accuracy? Which classes have higher accuracy compared to the others? Can you think of any reasons why? (Hint: Check where the errors in these classes are in the confusion matrix—i.e., being committed and omitted.)</p>
|
||||
<div class="callout-note callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
@@ -1265,8 +1290,8 @@ Note
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="hyperparameter-tuning" class="level3" data-number="2.3.2">
|
||||
<h3 data-number="2.3.2" class="anchored" data-anchor-id="hyperparameter-tuning"><span class="header-section-number">2.3.2</span> Hyperparameter tuning</h3>
|
||||
<section id="hyperparameter-tuning" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="hyperparameter-tuning">Hyperparameter tuning</h3>
|
||||
<p>We can also assess how the number of trees in the Random Forest classifier affects the classification accuracy. Copy and paste the code below to create a function that charts the overall accuracy versus the number of trees used. The code tests from 5 to 100 trees at increments of 5, producing Fig. F2.2.2. (Do not worry too much about fully understanding each item at this stage of your learning. If you want to find out how these operations work, you can see more in Chaps. F4.0 and F4.1.)</p>
|
||||
<div class="sourceCode" id="cb26"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb26-1"><a href="#cb26-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Hyperparameter tuning. </span></span>
|
||||
<span id="cb26-2"><a href="#cb26-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> numTrees <span class="op">=</span> ee<span class="op">.</span><span class="at">List</span><span class="op">.</span><span class="fu">sequence</span>(<span class="dv">5</span><span class="op">,</span> <span class="dv">100</span><span class="op">,</span> <span class="dv">5</span>)<span class="op">;</span> </span>
|
||||
@@ -1292,10 +1317,11 @@ Note
|
||||
<span id="cb26-22"><a href="#cb26-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">vAxis</span><span class="op">:</span> { </span>
|
||||
<span id="cb26-23"><a href="#cb26-23" aria-hidden="true" tabindex="-1"></a> <span class="dt">title</span><span class="op">:</span> <span class="st">'Accuracy'</span> }<span class="op">,</span> </span>
|
||||
<span id="cb26-24"><a href="#cb26-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">title</span><span class="op">:</span> <span class="st">'Accuracy per number of trees'</span> </span>
|
||||
<span id="cb26-25"><a href="#cb26-25" aria-hidden="true" tabindex="-1"></a>}))<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb26-25"><a href="#cb26-25" aria-hidden="true" tabindex="-1"></a>}))<span class="op">;</span></span>
|
||||
<span id="cb26-26"><a href="#cb26-26" aria-hidden="true" tabindex="-1"></a></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F2/image45.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F2/image45.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F2.2.2 Chart showing accuracy per number of Random Forest trees</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1596,13 +1622,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./F1.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></span>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Getting Started</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./F4.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/B3_Image_Series.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Image Series</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
File diff suppressed because it is too large
Load Diff
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - 4 Vectors and Tables</title>
|
||||
<title>Remote Sensing for OSINT - 6 Vectors and Tables</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -86,27 +86,27 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./lights.html" rel="next">
|
||||
<link href="./F4.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/C1_Lights.html" rel="next">
|
||||
<link href="../chapters/B3_Image_Series.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -146,7 +146,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Vectors and Tables</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -158,24 +158,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -218,17 +218,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -243,22 +243,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link active"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -273,27 +273,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -307,47 +307,47 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<h2 id="toc-title">Table of contents</h2>
|
||||
|
||||
<ul>
|
||||
<li><a href="#exploring-vectors" id="toc-exploring-vectors" class="nav-link active" data-scroll-target="#exploring-vectors"><span class="toc-section-number">4.1</span> Exploring Vectors</a>
|
||||
<li><a href="#exploring-vectors" id="toc-exploring-vectors" class="nav-link active" data-scroll-target="#exploring-vectors">Exploring Vectors</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#using-geometry-tools-to-create-features-in-earth-engine" id="toc-using-geometry-tools-to-create-features-in-earth-engine" class="nav-link" data-scroll-target="#using-geometry-tools-to-create-features-in-earth-engine"><span class="toc-section-number">4.1.1</span> Using Geometry Tools to Create Features in Earth Engine</a></li>
|
||||
<li><a href="#loading-existing-features-and-feature-collections-in-earth-engine" id="toc-loading-existing-features-and-feature-collections-in-earth-engine" class="nav-link" data-scroll-target="#loading-existing-features-and-feature-collections-in-earth-engine"><span class="toc-section-number">4.1.2</span> Loading Existing Features and Feature Collections in Earth Engine</a></li>
|
||||
<li><a href="#importing-features-into-earth-engine" id="toc-importing-features-into-earth-engine" class="nav-link" data-scroll-target="#importing-features-into-earth-engine"><span class="toc-section-number">4.1.3</span> Importing Features into Earth Engine</a></li>
|
||||
<li><a href="#filtering-feature-collections-by-attributes" id="toc-filtering-feature-collections-by-attributes" class="nav-link" data-scroll-target="#filtering-feature-collections-by-attributes"><span class="toc-section-number">4.1.4</span> Filtering Feature Collections by Attributes</a></li>
|
||||
<li><a href="#reducing-images-using-feature-geometry" id="toc-reducing-images-using-feature-geometry" class="nav-link" data-scroll-target="#reducing-images-using-feature-geometry"><span class="toc-section-number">4.1.5</span> Reducing Images Using Feature Geometry</a></li>
|
||||
<li><a href="#identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi" id="toc-identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi" class="nav-link" data-scroll-target="#identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi"><span class="toc-section-number">4.1.6</span> Identifying the Block in the Neighborhood Surrounding USF with the Highest NDVI</a></li>
|
||||
<li><a href="#using-geometry-tools-to-create-features-in-earth-engine" id="toc-using-geometry-tools-to-create-features-in-earth-engine" class="nav-link" data-scroll-target="#using-geometry-tools-to-create-features-in-earth-engine">Using Geometry Tools to Create Features in Earth Engine</a></li>
|
||||
<li><a href="#loading-existing-features-and-feature-collections-in-earth-engine" id="toc-loading-existing-features-and-feature-collections-in-earth-engine" class="nav-link" data-scroll-target="#loading-existing-features-and-feature-collections-in-earth-engine">Loading Existing Features and Feature Collections in Earth Engine</a></li>
|
||||
<li><a href="#importing-features-into-earth-engine" id="toc-importing-features-into-earth-engine" class="nav-link" data-scroll-target="#importing-features-into-earth-engine">Importing Features into Earth Engine</a></li>
|
||||
<li><a href="#filtering-feature-collections-by-attributes" id="toc-filtering-feature-collections-by-attributes" class="nav-link" data-scroll-target="#filtering-feature-collections-by-attributes">Filtering Feature Collections by Attributes</a></li>
|
||||
<li><a href="#reducing-images-using-feature-geometry" id="toc-reducing-images-using-feature-geometry" class="nav-link" data-scroll-target="#reducing-images-using-feature-geometry">Reducing Images Using Feature Geometry</a></li>
|
||||
<li><a href="#identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi" id="toc-identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi" class="nav-link" data-scroll-target="#identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi">Identifying the Block in the Neighborhood Surrounding USF with the Highest NDVI</a></li>
|
||||
<li><a href="#conclusion" id="toc-conclusion" class="nav-link" data-scroll-target="#conclusion">Conclusion</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#rastervector-conversions" id="toc-rastervector-conversions" class="nav-link" data-scroll-target="#rastervector-conversions"><span class="toc-section-number">4.2</span> Raster/Vector Conversions</a>
|
||||
<li><a href="#rastervector-conversions" id="toc-rastervector-conversions" class="nav-link" data-scroll-target="#rastervector-conversions">Raster/Vector Conversions</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#raster-to-vector-conversion" id="toc-raster-to-vector-conversion" class="nav-link" data-scroll-target="#raster-to-vector-conversion"><span class="toc-section-number">4.2.1</span> Raster to Vector Conversion</a></li>
|
||||
<li><a href="#a-more-complex-example" id="toc-a-more-complex-example" class="nav-link" data-scroll-target="#a-more-complex-example"><span class="toc-section-number">4.2.2</span> 3. A More Complex Example</a></li>
|
||||
<li><a href="#vector-to-raster-conversion" id="toc-vector-to-raster-conversion" class="nav-link" data-scroll-target="#vector-to-raster-conversion"><span class="toc-section-number">4.2.3</span> Vector-to-Raster Conversion</a></li>
|
||||
<li><a href="#raster-to-vector-conversion" id="toc-raster-to-vector-conversion" class="nav-link" data-scroll-target="#raster-to-vector-conversion">Raster to Vector Conversion</a></li>
|
||||
<li><a href="#a-more-complex-example" id="toc-a-more-complex-example" class="nav-link" data-scroll-target="#a-more-complex-example">3. A More Complex Example</a></li>
|
||||
<li><a href="#vector-to-raster-conversion" id="toc-vector-to-raster-conversion" class="nav-link" data-scroll-target="#vector-to-raster-conversion">Vector-to-Raster Conversion</a></li>
|
||||
<li><a href="#conclusion-1" id="toc-conclusion-1" class="nav-link" data-scroll-target="#conclusion-1">Conclusion</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#zonal-statistics" id="toc-zonal-statistics" class="nav-link" data-scroll-target="#zonal-statistics"><span class="toc-section-number">4.3</span> Zonal Statistics</a>
|
||||
<li><a href="#zonal-statistics" id="toc-zonal-statistics" class="nav-link" data-scroll-target="#zonal-statistics">Zonal Statistics</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#functions" id="toc-functions" class="nav-link" data-scroll-target="#functions"><span class="toc-section-number">4.3.1</span> Functions</a></li>
|
||||
<li><a href="#point-collection-creation" id="toc-point-collection-creation" class="nav-link" data-scroll-target="#point-collection-creation"><span class="toc-section-number">4.3.2</span> Point Collection Creation</a></li>
|
||||
<li><a href="#neighborhood-statistic-examples" id="toc-neighborhood-statistic-examples" class="nav-link" data-scroll-target="#neighborhood-statistic-examples"><span class="toc-section-number">4.3.3</span> Neighborhood Statistic Examples</a></li>
|
||||
<li><a href="#additional-notes" id="toc-additional-notes" class="nav-link" data-scroll-target="#additional-notes"><span class="toc-section-number">4.3.4</span> Additional Notes</a></li>
|
||||
<li><a href="#functions" id="toc-functions" class="nav-link" data-scroll-target="#functions">Functions</a></li>
|
||||
<li><a href="#point-collection-creation" id="toc-point-collection-creation" class="nav-link" data-scroll-target="#point-collection-creation">Point Collection Creation</a></li>
|
||||
<li><a href="#neighborhood-statistic-examples" id="toc-neighborhood-statistic-examples" class="nav-link" data-scroll-target="#neighborhood-statistic-examples">Neighborhood Statistic Examples</a></li>
|
||||
<li><a href="#additional-notes" id="toc-additional-notes" class="nav-link" data-scroll-target="#additional-notes">Additional Notes</a></li>
|
||||
<li><a href="#conclusion-2" id="toc-conclusion-2" class="nav-link" data-scroll-target="#conclusion-2">Conclusion</a></li>
|
||||
<li><a href="#references" id="toc-references" class="nav-link" data-scroll-target="#references">References</a></li>
|
||||
</ul></li>
|
||||
<li><a href="#advanced-vector-operations" id="toc-advanced-vector-operations" class="nav-link" data-scroll-target="#advanced-vector-operations"><span class="toc-section-number">4.4</span> Advanced Vector Operations</a>
|
||||
<li><a href="#advanced-vector-operations" id="toc-advanced-vector-operations" class="nav-link" data-scroll-target="#advanced-vector-operations">Advanced Vector Operations</a>
|
||||
<ul class="collapse">
|
||||
<li><a href="#visualizing-feature-collections" id="toc-visualizing-feature-collections" class="nav-link" data-scroll-target="#visualizing-feature-collections"><span class="toc-section-number">4.4.1</span> Visualizing Feature Collections</a></li>
|
||||
<li><a href="#joins-with-feature-collections" id="toc-joins-with-feature-collections" class="nav-link" data-scroll-target="#joins-with-feature-collections"><span class="toc-section-number">4.4.2</span> Joins with Feature Collections</a></li>
|
||||
<li><a href="#visualizing-feature-collections" id="toc-visualizing-feature-collections" class="nav-link" data-scroll-target="#visualizing-feature-collections">Visualizing Feature Collections</a></li>
|
||||
<li><a href="#joins-with-feature-collections" id="toc-joins-with-feature-collections" class="nav-link" data-scroll-target="#joins-with-feature-collections">Joins with Feature Collections</a></li>
|
||||
<li><a href="#conclusion-3" id="toc-conclusion-3" class="nav-link" data-scroll-target="#conclusion-3">Conclusion</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/F5.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/B4_Vectors_Tables.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Vectors and Tables</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -363,8 +363,8 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</header>
|
||||
|
||||
<p>In addition to raster data processing, Earth Engine supports a rich set of vector processing tools. This Part introduces you to the vector framework in Earth Engine, shows you how to create and to import your vector data, and how to combine vector and raster data for analyses.</p>
|
||||
<section id="exploring-vectors" class="level2" data-number="4.1">
|
||||
<h2 data-number="4.1" class="anchored" data-anchor-id="exploring-vectors"><span class="header-section-number">4.1</span> Exploring Vectors</h2>
|
||||
<section id="exploring-vectors" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="exploring-vectors">Exploring Vectors</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -412,13 +412,13 @@ Chapter Information
|
||||
<p>As you have seen, raster features in Earth Engine are stored as an Image or as part of an ImageCollection. Using a similar conceptual model, vector data in Earth Engine is stored as a Feature or as part of a FeatureCollection. Features and feature collections provide useful data to filter images and image collections by their location, clip images to a boundary, or statistically summarize the pixel values within a region.</p>
|
||||
<p>In the following example, you will use features and feature collections to identify which city block near the University of San Francisco (USF) campus is the most green.</p>
|
||||
</section>
|
||||
<section id="using-geometry-tools-to-create-features-in-earth-engine" class="level3" data-number="4.1.1">
|
||||
<h3 data-number="4.1.1" class="anchored" data-anchor-id="using-geometry-tools-to-create-features-in-earth-engine"><span class="header-section-number">4.1.1</span> Using Geometry Tools to Create Features in Earth Engine</h3>
|
||||
<section id="using-geometry-tools-to-create-features-in-earth-engine" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="using-geometry-tools-to-create-features-in-earth-engine">Using Geometry Tools to Create Features in Earth Engine</h3>
|
||||
<p>To demonstrate how geometry tools in Earth Engine work, let’s start by creating a point, and two polygons to represent different elements on the USF campus.</p>
|
||||
<p>Click on the geometry tools in the top left of the Map pane and create a point feature. Place a new point where USF is located (see Fig. F5.0.1).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image54.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image54.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.0.1 Location of the USF campus in San Francisco, California. Your first point should be in this vicinity. The red arrow points to the geometry tools.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -427,7 +427,7 @@ Chapter Information
|
||||
<p>After you create these layers, rename the geometry imports at the top of your script. Name the layers usf_point, usf_building, and usf_campus. These names are used within the script shown in Fig. F5.0.2.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image10.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image10.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.0.2 Rename the default variable names for each layer in the Imports section of the code at the top of your script</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -445,8 +445,8 @@ Note
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="loading-existing-features-and-feature-collections-in-earth-engine" class="level3" data-number="4.1.2">
|
||||
<h3 data-number="4.1.2" class="anchored" data-anchor-id="loading-existing-features-and-feature-collections-in-earth-engine"><span class="header-section-number">4.1.2</span> Loading Existing Features and Feature Collections in Earth Engine</h3>
|
||||
<section id="loading-existing-features-and-feature-collections-in-earth-engine" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="loading-existing-features-and-feature-collections-in-earth-engine">Loading Existing Features and Feature Collections in Earth Engine</h3>
|
||||
<p>If you wish to have the exact same geometry imports in this chapter for the rest of this exercise, begin this section using the code at the Code Checkpoint above.</p>
|
||||
<p>Next, you will load a city block dataset to determine the amount of vegetation on blocks near USF. The code below imports an existing feature dataset in Earth Engine. The Topologically Integrated Geographic Encoding and Referencing (TIGER) boundaries are census-designated boundaries that are a useful resource when comparing socioeconomic and diversity metrics with environmental datasets in the United States.</p>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Import the Census Tiger Boundaries from GEE. </span></span>
|
||||
@@ -456,37 +456,37 @@ Note
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(tiger<span class="op">,</span> { <span class="st">'color'</span><span class="op">:</span> <span class="st">'black'</span>}<span class="op">,</span> <span class="st">'Tiger'</span><span class="op">,</span> <span class="kw">false</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>You should now have the geometry for USF’s campus and a layer added to your map that is not visualized for census blocks across the United States. Next, we will use neighborhood data to spatially filter the TIGER feature collection for blocks near USF’s campus.</p>
|
||||
</section>
|
||||
<section id="importing-features-into-earth-engine" class="level3" data-number="4.1.3">
|
||||
<h3 data-number="4.1.3" class="anchored" data-anchor-id="importing-features-into-earth-engine"><span class="header-section-number">4.1.3</span> Importing Features into Earth Engine</h3>
|
||||
<section id="importing-features-into-earth-engine" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="importing-features-into-earth-engine">Importing Features into Earth Engine</h3>
|
||||
<p>There are many image collections loaded in Earth Engine, and they can cover a very large area that you might want to study. Borders can be quite intricate (for example, a detailed coastline), and fortunately there is no need for you to digitize the intricate boundary of a large geographic area. Instead, we will show how to find a spatial dataset online, download the data, and load this into Earth Engine as an asset for use.</p>
|
||||
<section id="find-a-spatial-dataset-of-san-francisco-neighborhoods" class="level4" data-number="4.1.3.1">
|
||||
<h4 data-number="4.1.3.1" class="anchored" data-anchor-id="find-a-spatial-dataset-of-san-francisco-neighborhoods"><span class="header-section-number">4.1.3.1</span> Find a Spatial Dataset of San Francisco Neighborhoods</h4>
|
||||
<section id="find-a-spatial-dataset-of-san-francisco-neighborhoods" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="find-a-spatial-dataset-of-san-francisco-neighborhoods">Find a Spatial Dataset of San Francisco Neighborhoods</h4>
|
||||
<p>Use your internet searching skills to locate the “Analysis Neighborhoods” dataset covering San Francisco. This data might be located in a number of places, including DataSF, the City of San Francisco’s public-facing data repository.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image27.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image27.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.0.3 DataSF website neighborhood shapefile to download</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>After you find the Analysis Neighborhoods layer, click Export and select Shapefile (Fig. F5.0.3). Keep track of where you save the zipped file, as we will load this into Earth Engine. Shapefiles contain vector-based data—points, lines, polygons—and include a number of files, such as the location information, attribute information, and others.</p>
|
||||
<p>Extract the folder to your computer. When you open the folder, you will see that there are actually many files. The extensions (.shp, .dbf, .shx, .prj) all provide a different piece of information to display vector-based data. The .shp file provides data on the geometry. The .dbf file provides data about the attributes. The .shx file is an index file. Lastly, the .prj file describes the map projection of the coordinate information for the shapefile. You will need to load all four files to create a new feature asset in Earth Engine.</p>
|
||||
</section>
|
||||
<section id="upload-sf-neighborhoods-file-as-an-asset" class="level4" data-number="4.1.3.2">
|
||||
<h4 data-number="4.1.3.2" class="anchored" data-anchor-id="upload-sf-neighborhoods-file-as-an-asset"><span class="header-section-number">4.1.3.2</span> Upload SF Neighborhoods File as an Asset</h4>
|
||||
<section id="upload-sf-neighborhoods-file-as-an-asset" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="upload-sf-neighborhoods-file-as-an-asset">Upload SF Neighborhoods File as an Asset</h4>
|
||||
<p>Navigate to the Assets tab (near Scripts). Select New > Table Upload > Shape files (Fig. F5.0.4).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image52.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image52.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.0.4 Import an asset as a zipped folder</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
<section id="select-files-and-name-asset" class="level4" data-number="4.1.3.3">
|
||||
<h4 data-number="4.1.3.3" class="anchored" data-anchor-id="select-files-and-name-asset"><span class="header-section-number">4.1.3.3</span> Select Files and Name Asset</h4>
|
||||
<section id="select-files-and-name-asset" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="select-files-and-name-asset">Select Files and Name Asset</h4>
|
||||
<p>Click the Select button and then use the file navigator to select the component files of the shapefile structure (i.e., .shp, .dbf, .shx, and .prj) (Fig. F5.0.5). Assign an Asset Name so you can recognize this asset.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image43.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image43.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.0.5 Select the four files extracted from the zipped folder. Make sure each file has the same name and that there are no spaces in the file names of the component files of the shapefile structure.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -515,10 +515,10 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="filtering-feature-collections-by-attributes" class="level3" data-number="4.1.4">
|
||||
<h3 data-number="4.1.4" class="anchored" data-anchor-id="filtering-feature-collections-by-attributes"><span class="header-section-number">4.1.4</span> Filtering Feature Collections by Attributes</h3>
|
||||
<section id="filter-by-geometry-of-another-feature" class="level4" data-number="4.1.4.1">
|
||||
<h4 data-number="4.1.4.1" class="anchored" data-anchor-id="filter-by-geometry-of-another-feature"><span class="header-section-number">4.1.4.1</span> Filter by Geometry of Another Feature</h4>
|
||||
<section id="filtering-feature-collections-by-attributes" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="filtering-feature-collections-by-attributes">Filtering Feature Collections by Attributes</h3>
|
||||
<section id="filter-by-geometry-of-another-feature" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="filter-by-geometry-of-another-feature">Filter by Geometry of Another Feature</h4>
|
||||
<p>First, let’s find the neighborhood associated with USF. Use the first point you created to find the neighborhood that intersects this point; filterBounds is the tool that does that, returning a filtered feature.</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Filter sfNeighborhoods by USF. </span></span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> usfNeighborhood <span class="op">=</span> sfNeighborhoods<span class="op">.</span><span class="fu">filterBounds</span>(usf_point)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
@@ -527,8 +527,8 @@ Note
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> usfTiger <span class="op">=</span> tiger<span class="op">.</span><span class="fu">filterBounds</span>(usfNeighborhood)<span class="op">;</span> </span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(usfTiger<span class="op">,</span> {}<span class="op">,</span> <span class="st">'usf_Tiger'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
<section id="filter-by-feature-attribute-properties" class="level4" data-number="4.1.4.2">
|
||||
<h4 data-number="4.1.4.2" class="anchored" data-anchor-id="filter-by-feature-attribute-properties"><span class="header-section-number">4.1.4.2</span> Filter by Feature (Attribute) Properties</h4>
|
||||
<section id="filter-by-feature-attribute-properties" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="filter-by-feature-attribute-properties">Filter by Feature (Attribute) Properties</h4>
|
||||
<p>In addition to filtering a FeatureCollection by the location of another feature, you can also filter it by its properties. First, let’s print the usfTiger variable to the Console and inspect the object.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(usfTiger)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>You can click on the feature collection name in the Console to uncover more information about the dataset. Click on the columns to learn about what attribute information is contained in this dataset. You will notice this feature collection contains information on both housing (‘housing10’) and population (‘pop10’).</p>
|
||||
@@ -543,8 +543,8 @@ Note
|
||||
<div class="sourceCode" id="cb8"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(housing10_g50_l250<span class="op">,</span> { <span class="st">'color'</span><span class="op">:</span> <span class="st">'Magenta'</span>}<span class="op">,</span> <span class="st">'housing'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>We have combined spatial and attribute information to narrow the set to only those blocks that meet our criteria of having between 50 and 250 housing units.</p>
|
||||
</section>
|
||||
<section id="print-feature-attribute-properties-to-console" class="level4" data-number="4.1.4.3">
|
||||
<h4 data-number="4.1.4.3" class="anchored" data-anchor-id="print-feature-attribute-properties-to-console"><span class="header-section-number">4.1.4.3</span> Print Feature (Attribute) Properties to Console</h4>
|
||||
<section id="print-feature-attribute-properties-to-console" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="print-feature-attribute-properties-to-console">Print Feature (Attribute) Properties to Console</h4>
|
||||
<p>We can print out attribute information about these features. The block of code below prints out the area of the resultant geometry in square meters.</p>
|
||||
<div class="sourceCode" id="cb9"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> housing_area <span class="op">=</span> housing10_g50_l250<span class="op">.</span><span class="fu">geometry</span>()<span class="op">.</span><span class="fu">area</span>()<span class="op">;</span> </span>
|
||||
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'housing_area:'</span><span class="op">,</span> housing_area)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
@@ -571,12 +571,12 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="reducing-images-using-feature-geometry" class="level3" data-number="4.1.5">
|
||||
<h3 data-number="4.1.5" class="anchored" data-anchor-id="reducing-images-using-feature-geometry"><span class="header-section-number">4.1.5</span> Reducing Images Using Feature Geometry</h3>
|
||||
<section id="reducing-images-using-feature-geometry" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="reducing-images-using-feature-geometry">Reducing Images Using Feature Geometry</h3>
|
||||
<p>Now that we have identified the blocks around USF’s campus that have the right housing density, let’s find which blocks are the greenest.</p>
|
||||
<p>The Normalized Difference Vegetation Index (NDVI), presented in detail in Chap. F2.0, is often used to compare the greenness of pixels in different locations. Values on land range from 0 to 1, with values closer to 1 representing healthier and greener vegetation than values near 0.</p>
|
||||
<section id="create-an-ndvi-image" class="level4" data-number="4.1.5.1">
|
||||
<h4 data-number="4.1.5.1" class="anchored" data-anchor-id="create-an-ndvi-image"><span class="header-section-number">4.1.5.1</span> Create an NDVI Image</h4>
|
||||
<section id="create-an-ndvi-image" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="create-an-ndvi-image">Create an NDVI Image</h4>
|
||||
<p>The code below imports the Landsat 8 ImageCollection as landsat8. Then, the code filters for images in 2021. Lastly, the code sorts the images from 2021 to find the least cloudy day.</p>
|
||||
<div class="sourceCode" id="cb11"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Import the Landsat 8 TOA image collection. </span></span>
|
||||
<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> landsat8 <span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">'LANDSAT/LC08/C02/T1_TOA'</span>)<span class="op">;</span> </span>
|
||||
@@ -593,8 +593,8 @@ Note
|
||||
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> red <span class="op">=</span> image<span class="op">.</span><span class="fu">select</span>(<span class="st">'B4'</span>)<span class="op">;</span> </span>
|
||||
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ndvi <span class="op">=</span> nir<span class="op">.</span><span class="fu">subtract</span>(red)<span class="op">.</span><span class="fu">divide</span>(nir<span class="op">.</span><span class="fu">add</span>(red))<span class="op">.</span><span class="fu">rename</span>(<span class="st">'NDVI'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
<section id="clip-the-ndvi-image-to-the-blocks-near-usf" class="level4" data-number="4.1.5.2">
|
||||
<h4 data-number="4.1.5.2" class="anchored" data-anchor-id="clip-the-ndvi-image-to-the-blocks-near-usf"><span class="header-section-number">4.1.5.2</span> Clip the NDVI Image to the Blocks Near USF</h4>
|
||||
<section id="clip-the-ndvi-image-to-the-blocks-near-usf" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="clip-the-ndvi-image-to-the-blocks-near-usf">Clip the NDVI Image to the Blocks Near USF</h4>
|
||||
<p>Next, you will clip the NDVI layer to only show NDVI over USF’s neighborhood.</p>
|
||||
<p>The first section of code provides visualization settings.</p>
|
||||
<div class="sourceCode" id="cb13"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ndviParams <span class="op">=</span> { </span>
|
||||
@@ -608,8 +608,8 @@ Note
|
||||
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">centerObject</span>(usf_point<span class="op">,</span> <span class="dv">14</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The NDVI map for all of San Francisco is interesting, and shows variability across the region. Now, let’s compute mean NDVI values for each block of the city.</p>
|
||||
</section>
|
||||
<section id="compute-ndvi-statistics-by-block" class="level4" data-number="4.1.5.3">
|
||||
<h4 data-number="4.1.5.3" class="anchored" data-anchor-id="compute-ndvi-statistics-by-block"><span class="header-section-number">4.1.5.3</span> Compute NDVI Statistics by Block</h4>
|
||||
<section id="compute-ndvi-statistics-by-block" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="compute-ndvi-statistics-by-block">Compute NDVI Statistics by Block</h4>
|
||||
<p>The code below uses the clipped image ndviUSFblocks and computes the mean NDVI value within each boundary. The scale provides a spatial resolution for the mean values to be computed on.</p>
|
||||
<div class="sourceCode" id="cb15"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Reduce image by feature to compute a statistic e.g. mean, max, min etc. </span></span>
|
||||
<span id="cb15-2"><a href="#cb15-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ndviPerBlock <span class="op">=</span> ndviUSFblocks<span class="op">.</span><span class="fu">reduceRegions</span>({ </span>
|
||||
@@ -619,8 +619,8 @@ Note
|
||||
<span id="cb15-6"><a href="#cb15-6" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now we’ll use Earth Engine to find out which block is greenest.</p>
|
||||
</section>
|
||||
<section id="export-table-of-ndvi-data-by-block-from-earth-engine-to-google-drive" class="level4" data-number="4.1.5.4">
|
||||
<h4 data-number="4.1.5.4" class="anchored" data-anchor-id="export-table-of-ndvi-data-by-block-from-earth-engine-to-google-drive"><span class="header-section-number">4.1.5.4</span> Export Table of NDVI Data by Block from Earth Engine to Google Drive</h4>
|
||||
<section id="export-table-of-ndvi-data-by-block-from-earth-engine-to-google-drive" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="export-table-of-ndvi-data-by-block-from-earth-engine-to-google-drive">Export Table of NDVI Data by Block from Earth Engine to Google Drive</h4>
|
||||
<p>Just as we loaded a feature into Earth Engine, we can export information from Earth Engine. Here, we will export the NDVI data, summarized by block, from Earth Engine to a Google Drive space so that we can interpret it in a program like Google Sheets or Excel.</p>
|
||||
<div class="sourceCode" id="cb16"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb16-1"><a href="#cb16-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Get a table of data out of Google Earth Engine. </span></span>
|
||||
<span id="cb16-2"><a href="#cb16-2" aria-hidden="true" tabindex="-1"></a>Export<span class="op">.</span><span class="at">table</span><span class="op">.</span><span class="fu">toDrive</span>({ </span>
|
||||
@@ -630,7 +630,7 @@ Note
|
||||
<p>When you run this code, you will notice that you have the Tasks tab highlighted on the top right of the Earth Engine Code Editor (Fig. F5.0.6). You will be prompted to name the directory when exporting the data.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image4.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image4.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.0.6 Under the Tasks tab, select Run to initiate download</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -650,8 +650,8 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi" class="level3" data-number="4.1.6">
|
||||
<h3 data-number="4.1.6" class="anchored" data-anchor-id="identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi"><span class="header-section-number">4.1.6</span> Identifying the Block in the Neighborhood Surrounding USF with the Highest NDVI</h3>
|
||||
<section id="identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi">Identifying the Block in the Neighborhood Surrounding USF with the Highest NDVI</h3>
|
||||
<p>You are already familiar with filtering datasets by their attributes. Now you will sort a table and select the first element of the table.</p>
|
||||
<div class="sourceCode" id="cb17"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb17-1"><a href="#cb17-1" aria-hidden="true" tabindex="-1"></a>ndviPerBlock <span class="op">=</span> ndviPerBlock<span class="op">.</span><span class="fu">select</span>([<span class="st">'blockid10'</span><span class="op">,</span> <span class="st">'mean'</span>])<span class="op">;</span> </span>
|
||||
<span id="cb17-2"><a href="#cb17-2" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'ndviPerBlock'</span><span class="op">,</span> ndviPerBlock)<span class="op">;</span> </span>
|
||||
@@ -686,8 +686,8 @@ Note
|
||||
<p>In this chapter, you learned how to import features into Earth Engine. In Sect. 1, you created new features using the geometry tools and loaded a feature from Earth Engine’s Data Catalog. In Sect. 2, you loaded a shapefile to an Earth Engine asset. In Sect. 3, you filtered feature collections based on their properties and locations. Finally, in Sects. 4 and 5, you used a feature collection to reduce an image, then exported the data from Earth Engine. Now you have all the tools you need to load, filter, and apply features to extract meaningful information from images using vector features in Earth Engine.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="rastervector-conversions" class="level2" data-number="4.2">
|
||||
<h2 data-number="4.2" class="anchored" data-anchor-id="rastervector-conversions"><span class="header-section-number">4.2</span> Raster/Vector Conversions</h2>
|
||||
<section id="rastervector-conversions" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="rastervector-conversions">Raster/Vector Conversions</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -735,10 +735,10 @@ Chapter Information
|
||||
<p>Raster and vector data are commonly combined (e.g., extracting image information for a given location or clipping an image to an area of interest); however, there are also situations in which conversion between the two formats is useful. In making such conversions, it is important to consider the key advantages of each format. Rasters can store data efficiently where each pixel has a numerical value, while vector data can more effectively represent geometric features where homogenous areas have shared properties. Each format lends itself to distinctive analytical operations, and combining them can be powerful.</p>
|
||||
<p>In this exercise, we’ll use topographic elevation and forest change images in Colombia as well as a protected area feature collection to practice the conversion between raster and vector formats, and to identify situations in which this is worthwhile.</p>
|
||||
</section>
|
||||
<section id="raster-to-vector-conversion" class="level3" data-number="4.2.1">
|
||||
<h3 data-number="4.2.1" class="anchored" data-anchor-id="raster-to-vector-conversion"><span class="header-section-number">4.2.1</span> Raster to Vector Conversion</h3>
|
||||
<section id="raster-to-polygons" class="level4" data-number="4.2.1.1">
|
||||
<h4 data-number="4.2.1.1" class="anchored" data-anchor-id="raster-to-polygons"><span class="header-section-number">4.2.1.1</span> Raster to Polygons</h4>
|
||||
<section id="raster-to-vector-conversion" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="raster-to-vector-conversion">Raster to Vector Conversion</h3>
|
||||
<section id="raster-to-polygons" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="raster-to-polygons">Raster to Polygons</h4>
|
||||
<p>In this section we will convert an elevation image (raster) to a feature collection (vector). We will start by loading the Global Multi-Resolution Terrain Elevation Data 2010 and the Global Administrative Unit Layers 2015 dataset to focus on Colombia. The elevation image is a raster at 7.5 arc-second spatial resolution containing a continuous measure of elevation in meters in each pixel.</p>
|
||||
<div class="sourceCode" id="cb19"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb19-1"><a href="#cb19-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Load raster (elevation) and vector (colombia) datasets. </span></span>
|
||||
<span id="cb19-2"><a href="#cb19-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> elevation <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>(<span class="st">'USGS/GMTED2010'</span>)<span class="op">.</span><span class="fu">rename</span>(<span class="st">'elevation'</span>)<span class="op">;</span> </span>
|
||||
@@ -788,12 +788,12 @@ Chapter Information
|
||||
<span id="cb21-19"><a href="#cb21-19" aria-hidden="true" tabindex="-1"></a> <span class="dt">strokeWidth</span><span class="op">:</span> <span class="dv">1</span> </span>
|
||||
<span id="cb21-20"><a href="#cb21-20" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span> </span>
|
||||
<span id="cb21-21"><a href="#cb21-21" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(elevationDrawn<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Elevation zone polygon'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="F5/image50.png" class="img-fluid"></p>
|
||||
<p><img src="F5/image33.png" class="img-fluid"></p>
|
||||
<p><img src="F5/image36.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image50.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image33.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image36.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image7.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image7.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.1 Raster-based elevation (top left) and zones (top right), vectorized elevation zones overlaid on the raster (bottom-left) and vectorized elevation zones only (bottom-right)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -826,21 +826,21 @@ Chapter Information
|
||||
<span id="cb22-26"><a href="#cb22-26" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span> </span>
|
||||
<span id="cb22-27"><a href="#cb22-27" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(smoothDrawn<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Elevation zone polygon (smooth)'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>We can see now that the polygons have more distinct shapes with many fewer small polygons in the new map (Fig. F5.1.2). It is important to note that when you use methods like focalMode (or other, similar methods such as connectedComponents and connectedPixelCount), you need to reproject according to the original image in order to display properly with zoom using the interactive Code Editor.</p>
|
||||
<p><img src="F5/image20.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image20.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image37.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image37.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.2 Before (left) and after (right) applying focalMode</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
<section id="raster-to-points" class="level4" data-number="4.2.1.2">
|
||||
<h4 data-number="4.2.1.2" class="anchored" data-anchor-id="raster-to-points"><span class="header-section-number">4.2.1.2</span> Raster to Points</h4>
|
||||
<section id="raster-to-points" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="raster-to-points">Raster to Points</h4>
|
||||
<p>Lastly, we will convert a small part of this elevation image into a point vector dataset. For this exercise, we will use the same example and build on the code from the previous subsection. This might be useful when you want to use geospatial data in a tabular format in combination with other conventional datasets such as economic indicators (Fig. F5.1.3).</p>
|
||||
<p><img src="F5/image24.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image24.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image11.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image11.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.3 Elevation point values with latitude and longitude</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -892,7 +892,7 @@ Chapter Information
|
||||
<span id="cb24-10"><a href="#cb24-10" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(elevationSamplesStratified<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Stratified samples'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image23.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image23.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.4 Stratified sampling over different elevation zones</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -911,8 +911,8 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="a-more-complex-example" class="level3" data-number="4.2.2">
|
||||
<h3 data-number="4.2.2" class="anchored" data-anchor-id="a-more-complex-example"><span class="header-section-number">4.2.2</span> 3. A More Complex Example</h3>
|
||||
<section id="a-more-complex-example" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="a-more-complex-example">3. A More Complex Example</h3>
|
||||
<p>In this section we’ll use two global datasets, one to represent raster formats and the other vectors:</p>
|
||||
<ul>
|
||||
<li>The Global Forest Change (GFC) dataset: a raster dataset describing global tree cover and change for 2001–present.</li>
|
||||
@@ -968,7 +968,7 @@ Note
|
||||
<p>This will display the boundary of the La Paya protected area and deforestation in the region (Fig. F5.1.5).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image55.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image55.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.5 View of the La Paya protected area in the Colombian Amazon (in white), and deforestation over the period 2001–2020 (in yellows and reds, with darker colors indicating more recent changes)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -995,10 +995,10 @@ Note
|
||||
<span id="cb26-20"><a href="#cb26-20" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb26-21"><a href="#cb26-21" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">20</span>}<span class="op">,</span> <span class="st">'Deforestation vector'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Fig. F5.1.6 shows a comparison of the raster versus vector representations of deforestation within the protected area.</p>
|
||||
<p><img src="F5/image42.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image42.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image13.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image13.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.6 Raster (left) versus vector (right) representations of deforestation data of the La Paya protected area</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1017,7 +1017,7 @@ Note
|
||||
<span id="cb27-12"><a href="#cb27-12" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span><span class="fu">print</span>(chart)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image15.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image15.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.7 Plot of the number of deforestation events in La Paya for the years 2001–2020</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1062,8 +1062,8 @@ Note
|
||||
<p>Code Checkpoint F51b. The book’s repository contains a script that shows what your code should look like at this point.</p>
|
||||
</div>
|
||||
</div>
|
||||
<section id="raster-properties-to-vector-fields" class="level4" data-number="4.2.2.1">
|
||||
<h4 data-number="4.2.2.1" class="anchored" data-anchor-id="raster-properties-to-vector-fields"><span class="header-section-number">4.2.2.1</span> Raster Properties to Vector Fields</h4>
|
||||
<section id="raster-properties-to-vector-fields" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="raster-properties-to-vector-fields">Raster Properties to Vector Fields</h4>
|
||||
<p>Sometimes we want to extract information from a raster to be included in an existing vector dataset. An example might be estimating a deforestation rate for a set of protected areas. Rather than perform this task on a case-by-case basis, we can attach information generated from an image as a property of a feature.</p>
|
||||
<p>The following script shows how this can be used to quantify a deforestation rate for a set of protected areas in the Colombian Amazon.</p>
|
||||
<div class="sourceCode" id="cb30"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb30-1"><a href="#cb30-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Load required datasets. </span></span>
|
||||
@@ -1135,11 +1135,11 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="vector-to-raster-conversion" class="level3" data-number="4.2.3">
|
||||
<h3 data-number="4.2.3" class="anchored" data-anchor-id="vector-to-raster-conversion"><span class="header-section-number">4.2.3</span> Vector-to-Raster Conversion</h3>
|
||||
<section id="vector-to-raster-conversion" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="vector-to-raster-conversion">Vector-to-Raster Conversion</h3>
|
||||
<p>In Sect. 1, we used the protected area feature collection as its original vector format. In this section, we will rasterize the protected area polygons to produce a mask and use this to assess rates of forest change.</p>
|
||||
<section id="polygons-to-a-mask" class="level4" data-number="4.2.3.1">
|
||||
<h4 data-number="4.2.3.1" class="anchored" data-anchor-id="polygons-to-a-mask"><span class="header-section-number">4.2.3.1</span> Polygons to a Mask</h4>
|
||||
<section id="polygons-to-a-mask" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="polygons-to-a-mask">Polygons to a Mask</h4>
|
||||
<p>The most common operation to convert from vector to raster is the production of binary image masks, describing whether a pixel intersects a line or falls within a polygon. To convert from vector to a raster mask, we can use the ee.FeatureCollection.reduceToImage method. Let’s continue with our example of the WDPA database and Global Forest Change data from the previous section:</p>
|
||||
<div class="sourceCode" id="cb32"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb32-1"><a href="#cb32-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Load required datasets. </span></span>
|
||||
<span id="cb32-2"><a href="#cb32-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> gfc <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>(<span class="st">'UMD/hansen/global_forest_change_2020_v1_8'</span>)<span class="op">;</span> </span>
|
||||
@@ -1200,8 +1200,8 @@ Note
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="a-more-complex-example-1" class="level4" data-number="4.2.3.2">
|
||||
<h4 data-number="4.2.3.2" class="anchored" data-anchor-id="a-more-complex-example-1"><span class="header-section-number">4.2.3.2</span> A More Complex Example</h4>
|
||||
<section id="a-more-complex-example-1" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="a-more-complex-example-1">A More Complex Example</h4>
|
||||
<p>The reduceToImage method is not the only way to convert a feature collection to an image. We will create a distance image layer from the boundary of the protected area using distance. For this example, we return to the La Paya protected area explored in Sect. 1.</p>
|
||||
<div class="sourceCode" id="cb35"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb35-1"><a href="#cb35-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Load required datasets. </span></span>
|
||||
<span id="cb35-2"><a href="#cb35-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> gfc <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>(<span class="st">'UMD/hansen/global_forest_change_2020_v1_8'</span>)<span class="op">;</span> </span>
|
||||
@@ -1233,11 +1233,11 @@ Note
|
||||
<span id="cb36-10"><a href="#cb36-10" aria-hidden="true" tabindex="-1"></a><span class="op">.</span><span class="fu">not</span>())<span class="op">,</span> { </span>
|
||||
<span id="cb36-11"><a href="#cb36-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> </span>
|
||||
<span id="cb36-12"><a href="#cb36-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">20000</span>}<span class="op">,</span> <span class="st">'Distance outside protected area'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="F5/image56.png" class="img-fluid"></p>
|
||||
<p><img src="F5/image9.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image56.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image9.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image25.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image25.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.8 Distance from the La Paya boundary (left), distance within the La Paya (middle), and distance outside the La Paya (right)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1267,12 +1267,12 @@ Note
|
||||
<span id="cb37-22"><a href="#cb37-22" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> </span>
|
||||
<span id="cb37-23"><a href="#cb37-23" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span> </span>
|
||||
<span id="cb37-24"><a href="#cb37-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">opacity</span><span class="op">:</span> <span class="fl">0.5</span>}<span class="op">,</span> <span class="st">'Deforestation within a 5km buffer'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="F5/image22.png" class="img-fluid"></p>
|
||||
<p><img src="F5/image6.png" class="img-fluid"></p>
|
||||
<p><img src="F5/image21.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image22.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image6.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image21.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image26.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image26.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.1.9 Distance zones (top left) and deforestation by zone (<1 km, <3 km, and <5 km)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1317,8 +1317,8 @@ Note
|
||||
<p>In this chapter, you learned how to convert raster to vector and vice versa. More importantly, you now have a better understanding of why and when such conversions are useful. Our examples should give you practical applications and ideas for using these techniques.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="zonal-statistics" class="level2" data-number="4.3">
|
||||
<h2 data-number="4.3" class="anchored" data-anchor-id="zonal-statistics"><span class="header-section-number">4.3</span> Zonal Statistics</h2>
|
||||
<section id="zonal-statistics" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="zonal-statistics">Zonal Statistics</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -1369,15 +1369,15 @@ Chapter Information
|
||||
<p>In fieldwork, researchers often work with plots, which are commonly recorded as polygon files or as a center point with a set radius. It is rare that plots will be set directly in the center of pixels from your desired raster dataset, and many field GPS units have positioning errors. Because of these issues, it may be important to use a statistic of adjacent pixels (as described in Chap. F3.2) to estimate the central value in what’s often called a neighborhood mean or focal mean (Cansler and McKenzie 2012, Miller and Thode 2007).</p>
|
||||
<p>To choose the size of your neighborhood, you will need to consider your research questions, the spatial resolution of the dataset, the size of your field plot, and the error from your GPS. For example, the raster value extracted for randomly placed 20 m diameter plots would likely merit use of a neighborhood mean when using Sentinel-2 or Landsat 8—at 10 m and 30 m spatial resolution, respectively—while using a thermal band from MODIS (Moderate Resolution Imaging Spectroradiometer) at 1000 m may not. While much of this tutorial is written with plot points and buffers in mind, a polygon asset with predefined regions will serve the same purpose.</p>
|
||||
</section>
|
||||
<section id="functions" class="level3" data-number="4.3.1">
|
||||
<h3 data-number="4.3.1" class="anchored" data-anchor-id="functions"><span class="header-section-number">4.3.1</span> Functions</h3>
|
||||
<section id="functions" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="functions">Functions</h3>
|
||||
<p>Two functions are provided; copy and paste them into your script:</p>
|
||||
<ul>
|
||||
<li>A function to generate circular or square regions from buffered points</li>
|
||||
<li>A function to extract image pixel neighborhood statistics for a given region</li>
|
||||
</ul>
|
||||
<section id="function-bufferpointsradius-bounds" class="level4" data-number="4.3.1.1">
|
||||
<h4 data-number="4.3.1.1" class="anchored" data-anchor-id="function-bufferpointsradius-bounds"><span class="header-section-number">4.3.1.1</span> Function: bufferPoints(radius, bounds)</h4>
|
||||
<section id="function-bufferpointsradius-bounds" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="function-bufferpointsradius-bounds">Function: bufferPoints(radius, bounds)</h4>
|
||||
<p>Our first function, bufferPoints, returns a function for adding a buffer to points and optionally transforming to rectangular bounds</p>
|
||||
<div class="sourceCode" id="cb39"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb39-1"><a href="#cb39-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">bufferPoints</span>(radius<span class="op">,</span> bounds) {</span>
|
||||
<span id="cb39-2"><a href="#cb39-2" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> <span class="kw">function</span>(pt) {</span>
|
||||
@@ -1387,8 +1387,8 @@ Chapter Information
|
||||
<span id="cb39-6"><a href="#cb39-6" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
|
||||
<span id="cb39-7"><a href="#cb39-7" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
<section id="function-zonalstatsfc-params" class="level4" data-number="4.3.1.2">
|
||||
<h4 data-number="4.3.1.2" class="anchored" data-anchor-id="function-zonalstatsfc-params"><span class="header-section-number">4.3.1.2</span> Function: zonalStats(fc, params)</h4>
|
||||
<section id="function-zonalstatsfc-params" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="function-zonalstatsfc-params">Function: zonalStats(fc, params)</h4>
|
||||
<p>The second function, zonalStats, reduces images in an ImageCollection by regions defined in a FeatureCollection. Note that reductions can return null statistics that you might want to filter out of the resulting feature collection. Null statistics occur when there are no valid pixels intersecting the region being reduced. This situation can be caused by points that are outside of an image or in regions that are masked for quality or clouds.</p>
|
||||
<p>This function is written to include many optional parameters (see Table F5.2.2). Look at the function carefully and note how it is written to include defaults that make it easy to apply the basic function while allowing customization.</p>
|
||||
<p>The desired datetime format. Use ISO 8601 data string standards. The datetime string is derived from the ‘system:time_start’ value of the ee.Image being reduced. Optional.</p>
|
||||
@@ -1466,8 +1466,8 @@ Chapter Information
|
||||
<span id="cb40-72"><a href="#cb40-72" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="point-collection-creation" class="level3" data-number="4.3.2">
|
||||
<h3 data-number="4.3.2" class="anchored" data-anchor-id="point-collection-creation"><span class="header-section-number">4.3.2</span> Point Collection Creation</h3>
|
||||
<section id="point-collection-creation" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="point-collection-creation">Point Collection Creation</h3>
|
||||
<p>Below, we create a set of points that form the basis of the zonal statistics calculations. Note that a unique plot_id property is added to each point. A unique plot or point ID is important to include in your vector dataset for future filtering and joining.</p>
|
||||
<div class="sourceCode" id="cb41"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb41-1"><a href="#cb41-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> pts <span class="op">=</span> ee<span class="op">.</span><span class="fu">FeatureCollection</span>([</span>
|
||||
<span id="cb41-2"><a href="#cb41-2" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="fu">Feature</span>(ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Point</span>([<span class="op">-</span><span class="fl">118.6010</span><span class="op">,</span> <span class="fl">37.0777</span>])<span class="op">,</span> {</span>
|
||||
@@ -1502,8 +1502,8 @@ Note
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<section id="neighborhood-statistic-examples" class="level3" data-number="4.3.3">
|
||||
<h3 data-number="4.3.3" class="anchored" data-anchor-id="neighborhood-statistic-examples"><span class="header-section-number">4.3.3</span> Neighborhood Statistic Examples</h3>
|
||||
<section id="neighborhood-statistic-examples" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="neighborhood-statistic-examples">Neighborhood Statistic Examples</h3>
|
||||
<p>The following examples demonstrate extracting raster neighborhood statistics for the following:</p>
|
||||
<ul>
|
||||
<li>A single raster with elevation and slope bands</li>
|
||||
@@ -1511,8 +1511,8 @@ Note
|
||||
<li>A multiband Landsat time series</li>
|
||||
</ul>
|
||||
<p>In each example, the points created in the previous section will be buffered and then used as regions to extract zonal statistics for each image in the image collection.</p>
|
||||
<section id="topographic-variables" class="level4" data-number="4.3.3.1">
|
||||
<h4 data-number="4.3.3.1" class="anchored" data-anchor-id="topographic-variables"><span class="header-section-number">4.3.3.1</span> Topographic Variables</h4>
|
||||
<section id="topographic-variables" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="topographic-variables">Topographic Variables</h4>
|
||||
<p>This example demonstrates how to calculate zonal statistics for a single multiband image. This Digital Elevation Model (DEM) contains a single topographic band representing elevation.</p>
|
||||
<p>####Buffer the Points</p>
|
||||
<p>Nex, we will apply a 45 m radius buffer to the points defined previously by mapping the bufferPoints function over the feature collection. The radius is set to 45 m to correspond to the 90 m pixel resolution of the DEM. In this case, circles are used instead of squares (set the second argument as false, i.e., do not use bounds).</p>
|
||||
@@ -1563,13 +1563,13 @@ Note
|
||||
<p>The result is a copy of the buffered point feature collection with new properties added for the region reduction of each selected image band according to the given reducer. A part of the FeatureCollection is shown in Fig. F5.2.1. The data in that FeatureCollection corresponds to a table containing the information of Table F5.2.3. See Fig. F5.2.2 for a graphical representation of the points and the topographic data being summarized.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image29.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image29.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.2.1 A part of the FeatureCollection produced by calculating the zonal statistics</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image5.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image5.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.2.2 Sample points and topographic slope. Elevation and slope values for regions intersecting each buffered point are reduced and attached as properties of the points.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1623,17 +1623,17 @@ Note
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
<section id="modis-time-series" class="level4" data-number="4.3.3.2">
|
||||
<h4 data-number="4.3.3.2" class="anchored" data-anchor-id="modis-time-series"><span class="header-section-number">4.3.3.2</span> MODIS Time Series</h4>
|
||||
<section id="modis-time-series" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="modis-time-series">MODIS Time Series</h4>
|
||||
<p>A time series of MODIS eight-day surface reflectance composites demonstrates how to calculate zonal statistics for a multiband ImageCollection that requires no preprocessing, such as cloud masking or computation. Note that there is no built-in function for performing region reductions on ImageCollection objects. The zonalStats function that we are using for reduction is mapping the reduceRegions function over an ImageCollection.</p>
|
||||
</section>
|
||||
<section id="buffer-the-points" class="level4" data-number="4.3.3.3">
|
||||
<h4 data-number="4.3.3.3" class="anchored" data-anchor-id="buffer-the-points"><span class="header-section-number">4.3.3.3</span> Buffer the Points</h4>
|
||||
<section id="buffer-the-points" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="buffer-the-points">Buffer the Points</h4>
|
||||
<p>In this example, suppose the point collection represents center points for field plots that are 100 m x 100 m, and apply a 50 m radius buffer to the points to match the size of the plot. Since we want zonal statistics for square plots, set the second argument of the bufferPoints function to true, so that the bounds of the buffered points are returned.</p>
|
||||
<p>var ptsModis = pts.map(bufferPoints(50, true));</p>
|
||||
</section>
|
||||
<section id="calculate-zonal-statistic" class="level4" data-number="4.3.3.4">
|
||||
<h4 data-number="4.3.3.4" class="anchored" data-anchor-id="calculate-zonal-statistic"><span class="header-section-number">4.3.3.4</span> Calculate Zonal Statistic</h4>
|
||||
<section id="calculate-zonal-statistic" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="calculate-zonal-statistic">Calculate Zonal Statistic</h4>
|
||||
<p>Import the MODIS 500 m global eight-day surface reflectance composite collection and filter the collection to include data for July, August, and September from 2015 through 2019.</p>
|
||||
<div class="sourceCode" id="cb45"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb45-1"><a href="#cb45-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> modisCol <span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">'MODIS/006/MOD09A1'</span>) </span>
|
||||
<span id="cb45-2"><a href="#cb45-2" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filterDate</span>(<span class="st">'2015-01-01'</span><span class="op">,</span> <span class="st">'2020-01-01'</span>) </span>
|
||||
@@ -1654,13 +1654,13 @@ Note
|
||||
<span id="cb46-13"><a href="#cb46-13" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> ptsModisStats <span class="op">=</span> <span class="fu">zonalStats</span>(modisCol<span class="op">,</span> ptsModis<span class="op">,</span> params)<span class="op">;</span><span class="fu">print</span>(<span class="st">'Limited MODIS zonal stats table'</span><span class="op">,</span> ptsModisStats<span class="op">.</span><span class="fu">limit</span>(<span class="dv">50</span>))<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The result is a feature collection with a feature for all combinations of plots and images. Interpreted as a table, the result has 200 rows (5 plots times 40 images) and as many columns as there are feature properties. Feature properties include those from the plot asset and the image, and any associated non-system image properties. Note that the printed results are limited to the first 50 features for brevity.</p>
|
||||
</section>
|
||||
<section id="landsat-time-series" class="level4" data-number="4.3.3.5">
|
||||
<h4 data-number="4.3.3.5" class="anchored" data-anchor-id="landsat-time-series"><span class="header-section-number">4.3.3.5</span> Landsat Time Series</h4>
|
||||
<section id="landsat-time-series" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="landsat-time-series">Landsat Time Series</h4>
|
||||
<p>This example combines Landsat surface reflectance imagery across three instruments: Thematic Mapper (TM) from Landsat 5, Enhanced Thematic Mapper Plus (ETM+) from Landsat 7, and Operational Land Imager (OLI) from Landsat 8.</p>
|
||||
<p>The following section prepares these collections so that band names are consistent and cloud masks are applied. Reflectance among corresponding bands are roughly congruent for the three sensors when using the surface reflectance product; therefore the processing steps that follow do not address inter-sensor harmonization. Review the current literature on inter-sensor harmonization practices if you’d like to apply a correction.</p>
|
||||
</section>
|
||||
<section id="prepare-the-landsat-image-collection" class="level4" data-number="4.3.3.6">
|
||||
<h4 data-number="4.3.3.6" class="anchored" data-anchor-id="prepare-the-landsat-image-collection"><span class="header-section-number">4.3.3.6</span> Prepare the Landsat Image Collection</h4>
|
||||
<section id="prepare-the-landsat-image-collection" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="prepare-the-landsat-image-collection">Prepare the Landsat Image Collection</h4>
|
||||
<p>First, define the function to mask cloud and shadow pixels (See Chap. F4.3 for more detail on cloud masking).</p>
|
||||
<div class="sourceCode" id="cb47"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb47-1"><a href="#cb47-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Mask clouds from images and apply scaling factors.</span></span>
|
||||
<span id="cb47-2"><a href="#cb47-2" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">maskScale</span>(img) {</span>
|
||||
@@ -1723,8 +1723,8 @@ Note
|
||||
<p>Merge the prepared sensor collections.</p>
|
||||
<div class="sourceCode" id="cb51"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb51-1"><a href="#cb51-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> landsatCol <span class="op">=</span> oliCol<span class="op">.</span><span class="fu">merge</span>(etmCol)<span class="op">.</span><span class="fu">merge</span>(tmCol)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
<section id="calculate-zonal-statistics" class="level4" data-number="4.3.3.7">
|
||||
<h4 data-number="4.3.3.7" class="anchored" data-anchor-id="calculate-zonal-statistics"><span class="header-section-number">4.3.3.7</span> Calculate Zonal Statistics</h4>
|
||||
<section id="calculate-zonal-statistics" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="calculate-zonal-statistics">Calculate Zonal Statistics</h4>
|
||||
<p>Reduce each image in the collection by each plot according to the following parameters. Note that this example defines the imgProps and imgPropsRename parameters to copy over and rename just two selected image properties: Landsat image ID and the satellite that collected the data. It also uses the max reducer, which, as an unweighted reducer, will return the maximum value from pixels that have their centroid within the buffer (see Sect. 4.1 below for more details).</p>
|
||||
<div class="sourceCode" id="cb52"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb52-1"><a href="#cb52-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Define parameters for the zonalStats function. </span></span>
|
||||
<span id="cb52-2"><a href="#cb52-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> params <span class="op">=</span> { </span>
|
||||
@@ -1744,8 +1744,8 @@ Note
|
||||
<span id="cb52-16"><a href="#cb52-16" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(<span class="st">'Limited Landsat zonal stats table'</span><span class="op">,</span> ptsLandsatStats<span class="op">.</span><span class="fu">limit</span>(<span class="dv">50</span>))<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The result is a feature collection with a feature for all combinations of plots and images.</p>
|
||||
</section>
|
||||
<section id="dealing-with-large-collections" class="level4" data-number="4.3.3.8">
|
||||
<h4 data-number="4.3.3.8" class="anchored" data-anchor-id="dealing-with-large-collections"><span class="header-section-number">4.3.3.8</span> Dealing with Large Collections</h4>
|
||||
<section id="dealing-with-large-collections" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="dealing-with-large-collections">Dealing with Large Collections</h4>
|
||||
<p>If your browser times out, try exporting the results (as described in Chap. F6.2). It’s likely that point feature collections that cover a large area or contain many points (point-image observations) will need to be exported as a batch task by either exporting the final feature collection as an asset or as a CSV/shapefile/GeoJSON to Google Drive or GCS.</p>
|
||||
<p>Here is how you would export the above Landsat image-point feature collection to an asset and to Google Drive. Run the following code, activate the Code Editor Tasks tab, and then click the Run button. If you don’t specify your own existing folder in Drive, the folder “EEFA_outputs” will be created.</p>
|
||||
<div class="sourceCode" id="cb53"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb53-1"><a href="#cb53-1" aria-hidden="true" tabindex="-1"></a>Export<span class="op">.</span><span class="at">table</span><span class="op">.</span><span class="fu">toAsset</span>({ </span>
|
||||
@@ -1774,15 +1774,15 @@ Note
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
<section id="additional-notes" class="level3" data-number="4.3.4">
|
||||
<h3 data-number="4.3.4" class="anchored" data-anchor-id="additional-notes"><span class="header-section-number">4.3.4</span> Additional Notes</h3>
|
||||
<section id="weighted-versus-unweighted-region-reduction" class="level4" data-number="4.3.4.1">
|
||||
<h4 data-number="4.3.4.1" class="anchored" data-anchor-id="weighted-versus-unweighted-region-reduction"><span class="header-section-number">4.3.4.1</span> Weighted Versus Unweighted Region Reduction</h4>
|
||||
<section id="additional-notes" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="additional-notes">Additional Notes</h3>
|
||||
<section id="weighted-versus-unweighted-region-reduction" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="weighted-versus-unweighted-region-reduction">Weighted Versus Unweighted Region Reduction</h4>
|
||||
<p>A region used for calculation of zonal statistics often bisects multiple pixels. Should partial pixels be included in zonal statistics? Earth Engine lets you decide by allowing you to define a reducer as either weighted or unweighted (or you can provide per-pixel weight specification as an image band). A weighted reducer will include partial pixels in the zonal statistic calculation by weighting each pixel’s contribution according to the fraction of the area intersecting the region. An unweighted reducer, on the other hand, gives equal weight to all pixels whose cell center intersects the region; all other pixels are excluded from calculation of the statistic.</p>
|
||||
<p>For aggregate reducers like ee.Reducer.mean and ee.Reducer.median, the default mode is weighted, while identifier reducers such as ee.Reducer.min and ee.Reducer.max are unweighted. You can adjust the behavior of weighted reducers by calling unweighted on them, as in ee.Reducer.mean.unweighted. You may also specify the weights by modifying the reducer with splitWeights; however, that is beyond the scope of this book.</p>
|
||||
</section>
|
||||
<section id="copy-properties-to-computed-images" class="level4" data-number="4.3.4.2">
|
||||
<h4 data-number="4.3.4.2" class="anchored" data-anchor-id="copy-properties-to-computed-images"><span class="header-section-number">4.3.4.2</span> Copy Properties to Computed Images</h4>
|
||||
<section id="copy-properties-to-computed-images" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="copy-properties-to-computed-images">Copy Properties to Computed Images</h4>
|
||||
<p>Derived, computed images do not retain the properties of their source image, so be sure to copy properties to computed images if you want them included in the region reduction table. For instance, consider the simple computation of unscaling Landsat SR data:</p>
|
||||
<div class="sourceCode" id="cb54"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb54-1"><a href="#cb54-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Define a Landsat image. </span></span>
|
||||
<span id="cb54-2"><a href="#cb54-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> img <span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">'LANDSAT/LC08/C02/T1_L2'</span>)<span class="op">.</span><span class="fu">first</span>()<span class="op">;</span> </span>
|
||||
@@ -1806,8 +1806,8 @@ Note
|
||||
<span id="cb55-8"><a href="#cb55-8" aria-hidden="true" tabindex="-1"></a><span class="op">.</span><span class="fu">propertyNames</span>())<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now selected properties are included. Use this technique when returning computed, derived images in a mapped function, and in single-image operations.</p>
|
||||
</section>
|
||||
<section id="understanding-which-pixels-are-included-in-polygon-statistics" class="level4" data-number="4.3.4.3">
|
||||
<h4 data-number="4.3.4.3" class="anchored" data-anchor-id="understanding-which-pixels-are-included-in-polygon-statistics"><span class="header-section-number">4.3.4.3</span> Understanding Which Pixels are Included in Polygon Statistics</h4>
|
||||
<section id="understanding-which-pixels-are-included-in-polygon-statistics" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="understanding-which-pixels-are-included-in-polygon-statistics">Understanding Which Pixels are Included in Polygon Statistics</h4>
|
||||
<p>If you want to visualize what pixels are included in a polygon for a region reducer, you can adapt the following code to use your own region (by replacing geometry), dataset, desired scale, and CRS parameters. The important part to note is that the image data you are adding to the map is reprojected using the same scale and CRS as that used in your region reduction (see Fig. F5.2.3).</p>
|
||||
<div class="sourceCode" id="cb56"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb56-1"><a href="#cb56-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Define polygon geometry. </span></span>
|
||||
<span id="cb56-2"><a href="#cb56-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> geometry <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>( </span>
|
||||
@@ -1867,7 +1867,7 @@ Note
|
||||
<span id="cb57-36"><a href="#cb57-36" aria-hidden="true" tabindex="-1"></a> <span class="dt">color</span><span class="op">:</span> <span class="st">'purple'</span>}<span class="op">,</span> <span class="st">'Pixels in reduction'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image44.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image44.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.2.3 Identifying pixels used in zonal statistics. By mapping the image and vector together, you can see which pixels are included in the unweighted statistic. For this example, three pixels would be included in the statistic because the polygon covers the center point of three pixels.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1903,8 +1903,8 @@ Note
|
||||
<p>Miller JD, Thode AE (2007) Quantifying burn severity in a heterogeneous landscape with a relative version of the delta Normalized Burn Ratio (dNBR). Remote Sens Environ 109:66–80. https://doi.org/10.1016/j.rse.2006.12.006</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="advanced-vector-operations" class="level2" data-number="4.4">
|
||||
<h2 data-number="4.4" class="anchored" data-anchor-id="advanced-vector-operations"><span class="header-section-number">4.4</span> Advanced Vector Operations</h2>
|
||||
<section id="advanced-vector-operations" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="advanced-vector-operations">Advanced Vector Operations</h2>
|
||||
<div class="callout-tip callout callout-style-default callout-captioned">
|
||||
<div class="callout-header d-flex align-content-center">
|
||||
<div class="callout-icon-container">
|
||||
@@ -1940,8 +1940,8 @@ Chapter Information
|
||||
</section>
|
||||
</div>
|
||||
</div>
|
||||
<section id="visualizing-feature-collections" class="level3" data-number="4.4.1">
|
||||
<h3 data-number="4.4.1" class="anchored" data-anchor-id="visualizing-feature-collections"><span class="header-section-number">4.4.1</span> Visualizing Feature Collections</h3>
|
||||
<section id="visualizing-feature-collections" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="visualizing-feature-collections">Visualizing Feature Collections</h3>
|
||||
<p>There is a distinct difference between how rasters and vectors are visualized. While images are typically visualized based on pixel values, vector layers use feature properties (i.e., attributes) to create a visualization. Vector layers are rendered on the Map by assigning a value to the red, green, and blue channels for each pixel on the screen based on the geometry and attributes of the features. The functions used for vector data visualization in Earth Engine are listed below in increasing order of complexity.</p>
|
||||
<ul>
|
||||
<li>Map.addLayer: As with raster layers, you can add a FeatureCollection to the Map by specifying visualization parameters. This method supports only one visualization parameter: color. All features are rendered with the specified color.</li>
|
||||
@@ -1950,8 +1950,8 @@ Chapter Information
|
||||
<li>style: This is the most versatile function. It can apply a different style to each feature, including color, pointSize, pointShape, width, fillColor, and lineType.</li>
|
||||
</ul>
|
||||
<p>In the exercises below, we will learn how to use each of these functions and see how they can generate different types of maps.</p>
|
||||
<section id="creating-a-choropleth-map" class="level4" data-number="4.4.1.1">
|
||||
<h4 data-number="4.4.1.1" class="anchored" data-anchor-id="creating-a-choropleth-map"><span class="header-section-number">4.4.1.1</span> Creating a Choropleth Map</h4>
|
||||
<section id="creating-a-choropleth-map" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="creating-a-choropleth-map">Creating a Choropleth Map</h4>
|
||||
<p>We will use the TIGER: US Census Blocks layer, which stores census block boundaries and their characteristics within the United States, along with the San Francisco neighborhoods layer from Chap. F5.0 to create a population density map for the city of San Francisco.</p>
|
||||
<p>We start by loading the census blocks and San Francisco neighborhoods layers. We use ee.Filter.bounds to filter the census blocks layer to the San Francisco boundary.</p>
|
||||
<div class="sourceCode" id="cb58"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb58-1"><a href="#cb58-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> blocks <span class="op">=</span> ee<span class="op">.</span><span class="fu">FeatureCollection</span>(<span class="st">'TIGER/2010/Blocks'</span>)<span class="op">;</span> </span>
|
||||
@@ -1969,7 +1969,7 @@ Chapter Information
|
||||
<span id="cb59-3"><a href="#cb59-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">color</span><span class="op">:</span> <span class="st">'#de2d26'</span>}<span class="op">,</span> <span class="st">'Census Blocks (single color)'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image34.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image34.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.1 San Francisco census blocks</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -1999,13 +1999,13 @@ Chapter Information
|
||||
<span id="cb62-7"><a href="#cb62-7" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(sfBlocksPaint<span class="op">.</span><span class="fu">clip</span>(geometry)<span class="op">,</span> visParams<span class="op">,</span> <span class="st">'Population Density'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image41.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image41.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.2 San Francisco population density</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
<section id="creating-a-categorical-map" class="level4" data-number="4.4.1.2">
|
||||
<h4 data-number="4.4.1.2" class="anchored" data-anchor-id="creating-a-categorical-map"><span class="header-section-number">4.4.1.2</span> Creating a Categorical Map</h4>
|
||||
<section id="creating-a-categorical-map" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="creating-a-categorical-map">Creating a Categorical Map</h4>
|
||||
<p>Continuing the exploration of styling methods, we will now learn about draw and style. These are the preferred methods of styling for points and line layers. Let’s see how we can visualize the TIGER: US Census Roads layer to create a categorical map.</p>
|
||||
<p>We start by filtering the roads layer to the San Francisco boundary and using Map.addLayer to visualize it.</p>
|
||||
<div class="sourceCode" id="cb63"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb63-1"><a href="#cb63-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Filter roads to San Francisco boundary. </span></span>
|
||||
@@ -2020,10 +2020,10 @@ Chapter Information
|
||||
<span id="cb64-4"><a href="#cb64-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">strokeWidth</span><span class="op">:</span> <span class="dv">1</span> </span>
|
||||
<span id="cb64-5"><a href="#cb64-5" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span> </span>
|
||||
<span id="cb64-6"><a href="#cb64-6" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(sfRoadsDraw<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Roads (Draw)'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="F5/image28.png" class="img-fluid"></p>
|
||||
<p><img src="../images/F5/image28.png" class="img-fluid"></p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image31.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image31.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.3 San Francisco roads rendered with a line width of 2 pixels (left) and and a line width of 1 pixel (right)</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2044,7 +2044,7 @@ Chapter Information
|
||||
<span id="cb67-4"><a href="#cb67-4" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(sfRoadsStyle<span class="op">.</span><span class="fu">clip</span>(geometry)<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Roads (Style)'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image46.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image46.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.4 San Francisco roads rendered according to road priority</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2064,8 +2064,8 @@ Note
|
||||
<p>Save your script for your own future use, as outlined in Chap. F1.0. Then, refresh the Code Editor to begin with a new script for the next section.</p>
|
||||
</section>
|
||||
</section>
|
||||
<section id="joins-with-feature-collections" class="level3" data-number="4.4.2">
|
||||
<h3 data-number="4.4.2" class="anchored" data-anchor-id="joins-with-feature-collections"><span class="header-section-number">4.4.2</span> Joins with Feature Collections</h3>
|
||||
<section id="joins-with-feature-collections" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="joins-with-feature-collections">Joins with Feature Collections</h3>
|
||||
<p>Earth Engine was designed as a platform for processing raster data, and that is where it shines. Over the years, it has acquired advanced vector data processing capabilities, and users are now able to carry out complex geoprocessing tasks within Earth Engine. You can leverage the distributed processing power of Earth Engine to process large vector layers in parallel.</p>
|
||||
<p>This section shows how you can do spatial queries and spatial joins using multiple large feature collections. This requires the use of joins. As described for Image Collections in Chap. F4.9, a join allows you to match every item in a collection with items in another collection based on certain conditions. While you can achieve similar results using map and filter, joins perform better and give you more flexibility. We need to define the following items to perform a join on two collections.</p>
|
||||
<ol type="1">
|
||||
@@ -2073,8 +2073,8 @@ Note
|
||||
<li>Join type: While the filter determines which features will be joined, the join type determines how they will be joined. There are many join types, including simple join, inner join, and save-all join.</li>
|
||||
</ol>
|
||||
<p>Joins are one of the harder skills to master, but doing so will help you perform many complex analysis tasks within Earth Engine. We will go through practical examples that will help you understand these concepts and the workflow better.</p>
|
||||
<section id="selecting-by-location" class="level4" data-number="4.4.2.1">
|
||||
<h4 data-number="4.4.2.1" class="anchored" data-anchor-id="selecting-by-location"><span class="header-section-number">4.4.2.1</span> Selecting by Location</h4>
|
||||
<section id="selecting-by-location" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="selecting-by-location">Selecting by Location</h4>
|
||||
<p>In this section, we will learn how to select features from one layer that are within a specified distance from features in another layer. We will continue to work with the San Francisco census blocks and roads datasets from the previous section. We will implement a join to select all blocks in San Francisco that are within 1 km of an interstate highway.</p>
|
||||
<p>We start by loading the census blocks and roads collections and filtering the roads layer to the San Francisco boundary.</p>
|
||||
<div class="sourceCode" id="cb68"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb68-1"><a href="#cb68-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> blocks <span class="op">=</span> ee<span class="op">.</span><span class="fu">FeatureCollection</span>(<span class="st">'TIGER/2010/Blocks'</span>)<span class="op">;</span> </span>
|
||||
@@ -2102,7 +2102,7 @@ Note
|
||||
<span id="cb70-10"><a href="#cb70-10" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(interstateRoadsDrawn<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Interstate Roads'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image2.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image2.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.5 San Francisco blocks and interstate highways</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2127,13 +2127,13 @@ Note
|
||||
<span id="cb73-5"><a href="#cb73-5" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(closeBlocksDrawn<span class="op">,</span> {}<span class="op">,</span> <span class="st">'Blocks within 1km'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image40.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image40.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.6 Selected blocks within 1 km of an interstate highway</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
</section>
|
||||
<section id="spatial-joins" class="level4" data-number="4.4.2.2">
|
||||
<h4 data-number="4.4.2.2" class="anchored" data-anchor-id="spatial-joins"><span class="header-section-number">4.4.2.2</span> Spatial Joins</h4>
|
||||
<section id="spatial-joins" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="spatial-joins">Spatial Joins</h4>
|
||||
<p>A spatial join allows you to query two collections based on the spatial relationship. We will now implement a spatial join to count points in polygons. We will work with a dataset of tree locations in San Francisco and polygons of neighborhoods to produce a CSV file with the total number of trees in each neighborhood.</p>
|
||||
<p>The San Francisco Open Data Portal maintains a street tree map dataset that has a list of street trees with their latitude and longitude. We will also use the San Francisco neighborhood dataset from the same portal. We downloaded, processed, and uploaded these layers as Earth Engine assets for use in this exercise. We start by loading both layers and using the paint and style functions, covered in Sect. 1, to visualize them (Fig. F5.3.7).</p>
|
||||
<p>var sfNeighborhoods = ee.FeatureCollection( ‘projects/gee-book/assets/F5-0/SFneighborhoods’);<br>
|
||||
@@ -2158,7 +2158,7 @@ var sfTrees = ee.FeatureCollection( ‘projects/gee-book/assets/F5-3/SFTrees’)
|
||||
<span id="cb74-18"><a href="#cb74-18" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(sfTreesStyled<span class="op">,</span> {}<span class="op">,</span> <span class="st">'SF Trees'</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image35.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image35.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.7 San Francisco neighborhoods and trees</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2178,7 +2178,7 @@ var sfTrees = ee.FeatureCollection( ‘projects/gee-book/assets/F5-3/SFTrees’)
|
||||
<span id="cb77-3"><a href="#cb77-3" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(joined<span class="op">.</span><span class="fu">first</span>())<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image1.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image1.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.8 Result of the save-all join</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2190,7 +2190,7 @@ var sfTrees = ee.FeatureCollection( ‘projects/gee-book/assets/F5-3/SFTrees’)
|
||||
<span id="cb78-5"><a href="#cb78-5" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(sfNeighborhoods<span class="op">.</span><span class="fu">first</span>())<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image18.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image18.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.9 Final FeatureCollection with the new property</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2208,7 +2208,7 @@ var sfTrees = ee.FeatureCollection( ‘projects/gee-book/assets/F5-3/SFTrees’)
|
||||
<p>The final result is a CSV file with the neighborhood names and total numbers of trees counted using the join (Fig. F5.3.10).</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="F5/image3.png" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/F5/image3.png" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Fig. F5.3.10 Exported CSV file with tree counts for San Francisco neighborhoods</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -2486,13 +2486,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./F4.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></span>
|
||||
<a href="../chapters/B3_Image_Series.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Image Series</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./lights.html" class="pagination-link">
|
||||
<span class="nav-page-text">War at Night</span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/C1_Lights.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">War at Night</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - War at Night</title>
|
||||
<title>Remote Sensing for OSINT - 7 War at Night</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -86,27 +86,27 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./refineries.html" rel="next">
|
||||
<link href="./F5.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/C2_Refineries.html" rel="next">
|
||||
<link href="../chapters/B4_Vectors_Tables.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -146,7 +146,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title">War at Night</h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">War at Night</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -158,24 +158,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -218,17 +218,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -243,22 +243,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -273,27 +273,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link active">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -313,14 +313,14 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<li><a href="#analysis" id="toc-analysis" class="nav-link" data-scroll-target="#analysis">Analysis</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/lights.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/C1_Lights.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block">War at Night</h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">War at Night</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -337,41 +337,45 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
|
||||
<section id="data" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="data">Data</h2>
|
||||
<p>Satellite images of Syria taken at night capture a subtle trace left by human civilization: lights. Apartment buildings, street lights, highways, powerplants– all are illuminated at night and can be seen from space. Researchers often use these nighttime lights signatures to track development; as cities grow, villages recieve power, and infrastructure is built, areas emit more light. But this works both ways. As cities are demolished, villages burned, and highways cutoff, they stop emitting lights.</p>
|
||||
<p>Satellite images of Syria taken at night capture a subtle trace left by human civilization: lights. Apartment buildings, street lights, highways, power plants – all are illuminated at night and can be seen from space. Researchers often use these nighttime lights signatures to track development; as cities grow, villages receive power and infrastructure is built, areas emit more light. But this works both ways. As cities are demolished, villages burned and highways cutoff, they stop emitting lights.</p>
|
||||
<p>In this tutorial, we’ll use satellite images of Iraq taken at night to track the destruction caused by the fight against the Islamic State. We’ll use the VIIRS nighttime lights dataset, which is a collection of satellite images taken by the Visible Infrared Imaging Radiometer Suite (VIIRS) on the Suomi NPP satellite. VIIRS is a sensor that can detect light in the visible and infrared spectrum, and is capable of taking images at night. A link to the GEE code for this section can be found <a href="https://code.earthengine.google.com/2cf77d8cb9afd76b73100637fbffdf5d">here</a>.</p>
|
||||
<section id="pre-processing" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="pre-processing">Pre-Processing</h3>
|
||||
<p>First, let’s start by importing a few useful packages written by <a href="https://twitter.com/gena_d">Gennadii Donchyts</a>. We’ll use <code>utils</code> and <code>text</code> to annotate the date of each image on the timelapse. We’ll also define an Area of Interest (AOI), which is just a rectangle. You can do this manually by clicking the drawing tools in the top left. I’ve drawn an AOI over the area covering Mosul, Irbil, and Kirkuk in Northern Iraq.</p>
|
||||
<p>First, let’s start by importing a few useful packages written by <a href="https://twitter.com/gena_d">Gennadii Donchyts</a>. We’ll use <code>utils</code> and <code>text</code> to annotate the date of each image on the timelapse. We’ll also define an Area of Interest (AOI), which is just a rectangle. You can do this manually by clicking the drawing tools in the top left. I’ve drawn an AOI over the area covering Mosul, Irbil and Kirkuk in Northern Iraq.</p>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> utils <span class="op">=</span> <span class="pp">require</span>(<span class="st">"users/gena/packages:utils"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> text <span class="op">=</span> <span class="pp">require</span>(<span class="st">"users/gena/packages:text"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="co">// define the Area of Interest (AOI)</span></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> AOI <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">42.555362833405326</span><span class="op">,</span> <span class="fl">36.62010778397765</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> [<span class="fl">42.555362833405326</span><span class="op">,</span> <span class="fl">35.18296243288332</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> [<span class="fl">44.681217325592826</span><span class="op">,</span> <span class="fl">35.18296243288332</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> [<span class="fl">44.681217325592826</span><span class="op">,</span> <span class="fl">36.62010778397765</span>]]])</span>
|
||||
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="co">// start and end dates for our gif </span></span>
|
||||
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> startDate <span class="op">=</span> <span class="st">'2013-01-01'</span><span class="op">;</span></span>
|
||||
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> endDate <span class="op">=</span> <span class="st">'2018-01-01'</span><span class="op">;</span></span>
|
||||
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="co">// a filename for when we export the gif</span></span>
|
||||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> export_name<span class="op">=</span><span class="st">'qayyarah_viirs'</span></span>
|
||||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a><span class="co">// A palette to visualize the VIIRS imagery. This one is similar to Matplotlib's "Magma" palette. </span></span>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> viirs_palette <span class="op">=</span> [</span>
|
||||
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> <span class="st">"#000004"</span><span class="op">,</span></span>
|
||||
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a> <span class="st">"#320a5a"</span><span class="op">,</span></span>
|
||||
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a> <span class="st">"#781b6c"</span><span class="op">,</span></span>
|
||||
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a> <span class="st">"#bb3654"</span><span class="op">,</span></span>
|
||||
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a> <span class="st">"#ec6824"</span><span class="op">,</span></span>
|
||||
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a> <span class="st">"#fbb41a"</span><span class="op">,</span></span>
|
||||
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a> <span class="st">"#fcffa4"</span><span class="op">,</span></span>
|
||||
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a>]<span class="op">;</span></span>
|
||||
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a><span class="co">// Visualisation parameters for the VIIRS imagery, defining a minimum and maximum value, and referencing the palette we just created</span></span>
|
||||
<span id="cb1-30"><a href="#cb1-30" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> VIIRSvis <span class="op">=</span> { <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="fl">0.1</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="fl">1.6</span><span class="op">,</span> <span class="dt">palette</span><span class="op">:</span> viirs_palette }<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="co">// define the Area of Interest (AOI)</span></span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> AOI <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">42.555362833405326</span><span class="op">,</span> <span class="fl">36.62010778397765</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a> [<span class="fl">42.555362833405326</span><span class="op">,</span> <span class="fl">35.18296243288332</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a> [<span class="fl">44.681217325592826</span><span class="op">,</span> <span class="fl">35.18296243288332</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a> [<span class="fl">44.681217325592826</span><span class="op">,</span> <span class="fl">36.62010778397765</span>]]])</span>
|
||||
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a><span class="co">// start and end dates for our gif </span></span>
|
||||
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> startDate <span class="op">=</span> <span class="st">'2013-01-01'</span><span class="op">;</span></span>
|
||||
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> endDate <span class="op">=</span> <span class="st">'2018-01-01'</span><span class="op">;</span></span>
|
||||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a><span class="co">// a filename for when we export the gif</span></span>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> export_name<span class="op">=</span><span class="st">'qayyarah_viirs'</span></span>
|
||||
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a><span class="co">// A palette to visualize the VIIRS imagery. This one is similar to Matplotlib's "Magma" palette. </span></span>
|
||||
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> viirs_palette <span class="op">=</span> [</span>
|
||||
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a> <span class="st">"#000004"</span><span class="op">,</span></span>
|
||||
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a> <span class="st">"#320a5a"</span><span class="op">,</span></span>
|
||||
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a> <span class="st">"#781b6c"</span><span class="op">,</span></span>
|
||||
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a> <span class="st">"#bb3654"</span><span class="op">,</span></span>
|
||||
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a> <span class="st">"#ec6824"</span><span class="op">,</span></span>
|
||||
<span id="cb1-28"><a href="#cb1-28" aria-hidden="true" tabindex="-1"></a> <span class="st">"#fbb41a"</span><span class="op">,</span></span>
|
||||
<span id="cb1-29"><a href="#cb1-29" aria-hidden="true" tabindex="-1"></a> <span class="st">"#fcffa4"</span><span class="op">,</span></span>
|
||||
<span id="cb1-30"><a href="#cb1-30" aria-hidden="true" tabindex="-1"></a>]<span class="op">;</span></span>
|
||||
<span id="cb1-31"><a href="#cb1-31" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-32"><a href="#cb1-32" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-33"><a href="#cb1-33" aria-hidden="true" tabindex="-1"></a><span class="co">// Visualisation parameters for the VIIRS imagery, defining a minimum and maximum value, and referencing the palette we just created</span></span>
|
||||
<span id="cb1-34"><a href="#cb1-34" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> VIIRSvis <span class="op">=</span> { <span class="dt">min</span><span class="op">:</span> <span class="op">-</span><span class="fl">0.1</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="fl">1.6</span><span class="op">,</span> <span class="dt">palette</span><span class="op">:</span> viirs_palette }<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Next, we’ll load the VIIRS nighttime lights imagery. We want to select the <code>avg_rad</code> band of the image collection, and filter blank images. Sometimes, we get blank images over an area in VIIRS if our AOI is on the edge of the satellite’s imaging swath. We can filter these images, similarly to how we filter for cloudy images in Sentinel-2:</p>
|
||||
<div class="sourceCode" id="cb2"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> VIIRS<span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">"NOAA/VIIRS/DNB/MONTHLY_V1/VCMCFG"</span>) </span>
|
||||
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">select</span>(<span class="st">'avg_rad'</span>)</span>
|
||||
@@ -396,8 +400,8 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setOptions</span>(<span class="st">'HYBRID'</span>)</span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">centerObject</span>(AOI)</span>
|
||||
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(VIIRS<span class="op">.</span><span class="fu">first</span>()<span class="op">,</span>VIIRSvis<span class="op">,</span><span class="st">'Nighttime Lights'</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="./images/iraq_check.png" class="img-fluid"></p>
|
||||
<p>If we decrease the opacity of the VIIRS layer, we can see the cities of Mosul, Erbil, and Kirkuk shining brightly at night. We can also see a string of bright lights between Kirkuk and Erbil– these are methane flares from oil wells.</p>
|
||||
<p><img src="../images/iraq_check.png" class="img-fluid"></p>
|
||||
<p>If we decrease the opacity of the VIIRS layer, we can see the cities of Mosul, Erbil and Kirkuk shining brightly at night. We can also see a string of bright lights between Kirkuk and Erbil – these are methane flares from oil wells.</p>
|
||||
</section>
|
||||
<section id="analysis" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="analysis">Analysis</h3>
|
||||
@@ -410,65 +414,71 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p>The function will then return a timelapse.</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> gif <span class="op">=</span> <span class="kw">function</span> (col<span class="op">,</span> col_vis<span class="op">,</span> AOI) {</span>
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> <span class="co">// Define the date annotations to be printed in the top left of the gif in white</span></span>
|
||||
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> annotations <span class="op">=</span> [</span>
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">textColor</span><span class="op">:</span> <span class="st">"white"</span><span class="op">,</span></span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">position</span><span class="op">:</span> <span class="st">"left"</span><span class="op">,</span></span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">offset</span><span class="op">:</span> <span class="st">"1%"</span><span class="op">,</span></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">margin</span><span class="op">:</span> <span class="st">"1%"</span><span class="op">,</span></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">property</span><span class="op">:</span> <span class="st">"label"</span><span class="op">,</span></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a> <span class="co">// Dynamically size the annotations according to the size of the AOI</span></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> AOI<span class="op">.</span><span class="fu">area</span>(<span class="dv">100</span>)<span class="op">.</span><span class="fu">sqrt</span>()<span class="op">.</span><span class="fu">divide</span>(<span class="dv">200</span>)<span class="op">,</span></span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> ]<span class="op">;</span></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a> <span class="co">// Next, we want to map over the image collection,</span></span>
|
||||
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> rgbVis <span class="op">=</span> col<span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span> (image) {</span>
|
||||
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the date of the image and format it</span></span>
|
||||
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> start <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(image<span class="op">.</span><span class="fu">get</span>(<span class="st">"system:time_start"</span>))<span class="op">;</span></span>
|
||||
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> label <span class="op">=</span> start<span class="op">.</span><span class="fu">format</span>(<span class="st">"YYYY-MM-dd"</span>)<span class="op">;</span></span>
|
||||
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a> <span class="co">// And visualize the image using the visualization parameters defined earlier.</span></span>
|
||||
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a> <span class="co">// We also want to set a property called "label" that stores the formatted date </span></span>
|
||||
<span id="cb4-23"><a href="#cb4-23" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> image<span class="op">.</span><span class="fu">visualize</span>(col_vis)<span class="op">.</span><span class="fu">set</span>({ <span class="dt">label</span><span class="op">:</span> label })<span class="op">;</span></span>
|
||||
<span id="cb4-24"><a href="#cb4-24" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-25"><a href="#cb4-25" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-26"><a href="#cb4-26" aria-hidden="true" tabindex="-1"></a> <span class="co">// Now we use the label proprty and the annotateImage function from @gena_d to annotate each image with the date. </span></span>
|
||||
<span id="cb4-27"><a href="#cb4-27" aria-hidden="true" tabindex="-1"></a> rgbVis <span class="op">=</span> rgbVis<span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span> (image) {</span>
|
||||
<span id="cb4-28"><a href="#cb4-28" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> text<span class="op">.</span><span class="fu">annotateImage</span>(image<span class="op">,</span> {}<span class="op">,</span> AOI<span class="op">,</span> annotations)<span class="op">;</span></span>
|
||||
<span id="cb4-29"><a href="#cb4-29" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-30"><a href="#cb4-30" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-31"><a href="#cb4-31" aria-hidden="true" tabindex="-1"></a> <span class="co">// Define GIF visualization parameters.</span></span>
|
||||
<span id="cb4-32"><a href="#cb4-32" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> gifParams <span class="op">=</span> {</span>
|
||||
<span id="cb4-33"><a href="#cb4-33" aria-hidden="true" tabindex="-1"></a> <span class="dt">maxPixels</span><span class="op">:</span> <span class="dv">27017280</span><span class="op">,</span></span>
|
||||
<span id="cb4-34"><a href="#cb4-34" aria-hidden="true" tabindex="-1"></a> <span class="dt">region</span><span class="op">:</span> AOI<span class="op">,</span></span>
|
||||
<span id="cb4-35"><a href="#cb4-35" aria-hidden="true" tabindex="-1"></a> <span class="dt">crs</span><span class="op">:</span> <span class="st">"EPSG:3857"</span><span class="op">,</span></span>
|
||||
<span id="cb4-36"><a href="#cb4-36" aria-hidden="true" tabindex="-1"></a> <span class="dt">dimensions</span><span class="op">:</span> <span class="dv">640</span><span class="op">,</span></span>
|
||||
<span id="cb4-37"><a href="#cb4-37" aria-hidden="true" tabindex="-1"></a> <span class="dt">framesPerSecond</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span></span>
|
||||
<span id="cb4-38"><a href="#cb4-38" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
|
||||
<span id="cb4-39"><a href="#cb4-39" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-40"><a href="#cb4-40" aria-hidden="true" tabindex="-1"></a> <span class="co">// Export the gif to Google Drive</span></span>
|
||||
<span id="cb4-41"><a href="#cb4-41" aria-hidden="true" tabindex="-1"></a> Export<span class="op">.</span><span class="at">video</span><span class="op">.</span><span class="fu">toDrive</span>({</span>
|
||||
<span id="cb4-42"><a href="#cb4-42" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> rgbVis<span class="op">,</span> <span class="co">// the image collection</span></span>
|
||||
<span id="cb4-43"><a href="#cb4-43" aria-hidden="true" tabindex="-1"></a> <span class="dt">description</span><span class="op">:</span> export_name<span class="op">,</span> <span class="co">// the name of the file</span></span>
|
||||
<span id="cb4-44"><a href="#cb4-44" aria-hidden="true" tabindex="-1"></a> <span class="dt">dimensions</span><span class="op">:</span> <span class="dv">1080</span><span class="op">,</span> <span class="co">// the dimensions of the gif</span></span>
|
||||
<span id="cb4-45"><a href="#cb4-45" aria-hidden="true" tabindex="-1"></a> <span class="dt">framesPerSecond</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span> <span class="co">// the number of frames per second</span></span>
|
||||
<span id="cb4-46"><a href="#cb4-46" aria-hidden="true" tabindex="-1"></a> <span class="dt">region</span><span class="op">:</span> AOI<span class="op">,</span> <span class="co">// the area of interest</span></span>
|
||||
<span id="cb4-47"><a href="#cb4-47" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-48"><a href="#cb4-48" aria-hidden="true" tabindex="-1"></a> <span class="co">// Print the GIF URL to the console.</span></span>
|
||||
<span id="cb4-49"><a href="#cb4-49" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(rgbVis<span class="op">.</span><span class="fu">getVideoThumbURL</span>(gifParams))<span class="op">;</span></span>
|
||||
<span id="cb4-50"><a href="#cb4-50" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-51"><a href="#cb4-51" aria-hidden="true" tabindex="-1"></a> <span class="co">// Render the GIF animation in the console.</span></span>
|
||||
<span id="cb4-52"><a href="#cb4-52" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(ui<span class="op">.</span><span class="fu">Thumbnail</span>(rgbVis<span class="op">,</span> gifParams))<span class="op">;</span></span>
|
||||
<span id="cb4-53"><a href="#cb4-53" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Ok that was a pretty big chunk of code. But the good news is that we basically never have to touch it again, since we can just feed it different inputs. For example, if I want to generate a gif of nighttime lights over a different area, it’s as simple as dragging the AOI. If I want to look at a different time period, I can just edit the <code>startDate</code> and <code>endDate</code> variables. And if I want to visualize an entirely different type of satellite imagery– Sentinel-1, Sentinel-2, or anything else, all I have to do is change the image collection (<code>col</code>) and visualization parameters (<code>col_vis</code>) variables. Now, let’s look at some timelapses.</p>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="co">// Define the date annotations to be printed in the top left of the gif in white</span></span>
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> annotations <span class="op">=</span> [</span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">textColor</span><span class="op">:</span> <span class="st">"white"</span><span class="op">,</span></span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">position</span><span class="op">:</span> <span class="st">"left"</span><span class="op">,</span></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">offset</span><span class="op">:</span> <span class="st">"1%"</span><span class="op">,</span></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">margin</span><span class="op">:</span> <span class="st">"1%"</span><span class="op">,</span></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">property</span><span class="op">:</span> <span class="st">"label"</span><span class="op">,</span></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Dynamically size the annotations according to the size of the AOI</span></span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> AOI<span class="op">.</span><span class="fu">area</span>(<span class="dv">100</span>)<span class="op">.</span><span class="fu">sqrt</span>()<span class="op">.</span><span class="fu">divide</span>(<span class="dv">200</span>)<span class="op">,</span></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> }<span class="op">,</span></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> ]<span class="op">;</span></span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a> <span class="co">// Next, we want to map over the image collection,</span></span>
|
||||
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> rgbVis <span class="op">=</span> col<span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span> (image) {</span>
|
||||
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the date of the image and format it</span></span>
|
||||
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> start <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(image<span class="op">.</span><span class="fu">get</span>(<span class="st">"system:time_start"</span>))<span class="op">;</span></span>
|
||||
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> label <span class="op">=</span> start<span class="op">.</span><span class="fu">format</span>(<span class="st">"YYYY-MM-dd"</span>)<span class="op">;</span></span>
|
||||
<span id="cb4-23"><a href="#cb4-23" aria-hidden="true" tabindex="-1"></a> <span class="co">// And visualize the image using the visualization parameters defined earlier.</span></span>
|
||||
<span id="cb4-24"><a href="#cb4-24" aria-hidden="true" tabindex="-1"></a> <span class="co">// We also want to set a property called "label" that stores the formatted date </span></span>
|
||||
<span id="cb4-25"><a href="#cb4-25" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> image<span class="op">.</span><span class="fu">visualize</span>(col_vis)<span class="op">.</span><span class="fu">set</span>({ <span class="dt">label</span><span class="op">:</span> label })<span class="op">;</span></span>
|
||||
<span id="cb4-26"><a href="#cb4-26" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-27"><a href="#cb4-27" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-28"><a href="#cb4-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-29"><a href="#cb4-29" aria-hidden="true" tabindex="-1"></a> <span class="co">// Now we use the label property and the annotateImage function from @gena_d to annotate each image with the date. </span></span>
|
||||
<span id="cb4-30"><a href="#cb4-30" aria-hidden="true" tabindex="-1"></a> rgbVis <span class="op">=</span> rgbVis<span class="op">.</span><span class="fu">map</span>(<span class="kw">function</span> (image) {</span>
|
||||
<span id="cb4-31"><a href="#cb4-31" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> text<span class="op">.</span><span class="fu">annotateImage</span>(image<span class="op">,</span> {}<span class="op">,</span> AOI<span class="op">,</span> annotations)<span class="op">;</span></span>
|
||||
<span id="cb4-32"><a href="#cb4-32" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-33"><a href="#cb4-33" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-34"><a href="#cb4-34" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-35"><a href="#cb4-35" aria-hidden="true" tabindex="-1"></a> <span class="co">// Define GIF visualization parameters.</span></span>
|
||||
<span id="cb4-36"><a href="#cb4-36" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> gifParams <span class="op">=</span> {</span>
|
||||
<span id="cb4-37"><a href="#cb4-37" aria-hidden="true" tabindex="-1"></a> <span class="dt">maxPixels</span><span class="op">:</span> <span class="dv">27017280</span><span class="op">,</span></span>
|
||||
<span id="cb4-38"><a href="#cb4-38" aria-hidden="true" tabindex="-1"></a> <span class="dt">region</span><span class="op">:</span> AOI<span class="op">,</span></span>
|
||||
<span id="cb4-39"><a href="#cb4-39" aria-hidden="true" tabindex="-1"></a> <span class="dt">crs</span><span class="op">:</span> <span class="st">"EPSG:3857"</span><span class="op">,</span></span>
|
||||
<span id="cb4-40"><a href="#cb4-40" aria-hidden="true" tabindex="-1"></a> <span class="dt">dimensions</span><span class="op">:</span> <span class="dv">640</span><span class="op">,</span></span>
|
||||
<span id="cb4-41"><a href="#cb4-41" aria-hidden="true" tabindex="-1"></a> <span class="dt">framesPerSecond</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span></span>
|
||||
<span id="cb4-42"><a href="#cb4-42" aria-hidden="true" tabindex="-1"></a> }<span class="op">;</span></span>
|
||||
<span id="cb4-43"><a href="#cb4-43" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-44"><a href="#cb4-44" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-45"><a href="#cb4-45" aria-hidden="true" tabindex="-1"></a> <span class="co">// Export the gif to Google Drive</span></span>
|
||||
<span id="cb4-46"><a href="#cb4-46" aria-hidden="true" tabindex="-1"></a> Export<span class="op">.</span><span class="at">video</span><span class="op">.</span><span class="fu">toDrive</span>({</span>
|
||||
<span id="cb4-47"><a href="#cb4-47" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> rgbVis<span class="op">,</span> <span class="co">// the image collection</span></span>
|
||||
<span id="cb4-48"><a href="#cb4-48" aria-hidden="true" tabindex="-1"></a> <span class="dt">description</span><span class="op">:</span> export_name<span class="op">,</span> <span class="co">// the name of the file</span></span>
|
||||
<span id="cb4-49"><a href="#cb4-49" aria-hidden="true" tabindex="-1"></a> <span class="dt">dimensions</span><span class="op">:</span> <span class="dv">1080</span><span class="op">,</span> <span class="co">// the dimensions of the gif</span></span>
|
||||
<span id="cb4-50"><a href="#cb4-50" aria-hidden="true" tabindex="-1"></a> <span class="dt">framesPerSecond</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span> <span class="co">// the number of frames per second</span></span>
|
||||
<span id="cb4-51"><a href="#cb4-51" aria-hidden="true" tabindex="-1"></a> <span class="dt">region</span><span class="op">:</span> AOI<span class="op">,</span> <span class="co">// the area of interest</span></span>
|
||||
<span id="cb4-52"><a href="#cb4-52" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-53"><a href="#cb4-53" aria-hidden="true" tabindex="-1"></a> <span class="co">// Print the GIF URL to the console.</span></span>
|
||||
<span id="cb4-54"><a href="#cb4-54" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(rgbVis<span class="op">.</span><span class="fu">getVideoThumbURL</span>(gifParams))<span class="op">;</span></span>
|
||||
<span id="cb4-55"><a href="#cb4-55" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-56"><a href="#cb4-56" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-57"><a href="#cb4-57" aria-hidden="true" tabindex="-1"></a> <span class="co">// Render the GIF animation in the console.</span></span>
|
||||
<span id="cb4-58"><a href="#cb4-58" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(ui<span class="op">.</span><span class="fu">Thumbnail</span>(rgbVis<span class="op">,</span> gifParams))<span class="op">;</span></span>
|
||||
<span id="cb4-59"><a href="#cb4-59" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Ok that was a pretty big chunk of code. But the good news is that we basically never have to touch it again, since we can just feed it different inputs. For example, if I want to generate a gif of night time lights over a different area, it’s as simple as dragging the AOI. If I want to look at a different time period, I can just edit the <code>startDate</code> and <code>endDate</code> variables. And if I want to visualize an entirely different type of satellite imagery – Sentinel-1, Sentinel-2, or anything else, all I have to do is change the image collection (<code>col</code>) and visualization parameters (<code>col_vis</code>) variables. Now, let’s look at some timelapses.</p>
|
||||
<section id="the-fall-of-mosul" class="level4">
|
||||
<h4 class="anchored" data-anchor-id="the-fall-of-mosul">The Fall of Mosul</h4>
|
||||
<p>The function returns a timelapse of nighttime lights over Northern Iraq:</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="fu">gif</span>(VIIRS<span class="op">,</span> VIIRSvis<span class="op">,</span> AOI)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/Figure_1.gif" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/Figure_1.gif" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">I’ve done a bit of post-processing to this gif, adding more annotations and blending between frames to make it a bit smoother. I typically use <a href="https://ffmpeg.org/">ffmpeg</a> and <a href="https://ezgif.com/">ezgif</a> for the finishing touches.</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
@@ -489,19 +499,21 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="st">"system:index"</span><span class="op">:</span> <span class="st">"0"</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> })<span class="op">,</span></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a> qayyarah <span class="op">=</span> ee<span class="op">.</span><span class="fu">Feature</span>(</span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">43.08240275545117</span><span class="op">,</span> <span class="fl">35.8925587996721</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.08240275545117</span><span class="op">,</span> <span class="fl">35.77899970860588</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.26642375154492</span><span class="op">,</span> <span class="fl">35.77899970860588</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.26642375154492</span><span class="op">,</span> <span class="fl">35.8925587996721</span>]]]<span class="op">,</span> <span class="kw">null</span><span class="op">,</span> <span class="kw">false</span>)<span class="op">,</span></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> <span class="st">"label"</span><span class="op">:</span> <span class="st">"Qayyarah"</span><span class="op">,</span></span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> <span class="st">"system:index"</span><span class="op">:</span> <span class="st">"0"</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> })</span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a><span class="co">// Let's put these together in a list </span></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> regions<span class="op">=</span>[qayyarah<span class="op">,</span> mosul]</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> qayyarah <span class="op">=</span> ee<span class="op">.</span><span class="fu">Feature</span>(</span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>(</span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> [[[<span class="fl">43.08240275545117</span><span class="op">,</span> <span class="fl">35.8925587996721</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.08240275545117</span><span class="op">,</span> <span class="fl">35.77899970860588</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.26642375154492</span><span class="op">,</span> <span class="fl">35.77899970860588</span>]<span class="op">,</span></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> [<span class="fl">43.26642375154492</span><span class="op">,</span> <span class="fl">35.8925587996721</span>]]]<span class="op">,</span> <span class="kw">null</span><span class="op">,</span> <span class="kw">false</span>)<span class="op">,</span></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> {</span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> <span class="st">"label"</span><span class="op">:</span> <span class="st">"Qayyarah"</span><span class="op">,</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> <span class="st">"system:index"</span><span class="op">:</span> <span class="st">"0"</span></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a> })</span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a><span class="co">// Let's put these together in a list </span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> regions<span class="op">=</span>[qayyarah<span class="op">,</span> mosul]</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Once we’ve got the rectangles, we can make a chart that will take the mean value of the VIIRS images in each rectangle over time:</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> chart <span class="op">=</span></span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a> ui<span class="op">.</span><span class="at">Chart</span><span class="op">.</span><span class="at">image</span></span>
|
||||
@@ -515,39 +527,53 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(chart)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p><img src="./images/qayyarah_chart.png" class="img-fluid"></p>
|
||||
<p>We can clearly see Mosul (the red line) darkening in 2014 as the city is taken by ISIS. During this period the Qayyarah oilfileds are, as we might expect, quite dark. All of a sudden in 2016 Qayyarah becomes brighter at night than the city of Mosul ever was, as the oilfields are set on fire. Then, almost exactly when the blaze in Qayyarah is extinguished and the area darkens (i.e. when the blue line falls back to near zero), Mosul brightens once again (i.e. the red line rises) as the city is liberated.</p>
|
||||
<p><img src="../images/qayyarah_chart.png" class="img-fluid"></p>
|
||||
<p>We can clearly see Mosul (the red line) darkening in 2014 as the city is taken by ISIS. During this period the Qayyarah oil fields are, as we might expect, quite dark. All of a sudden in 2016 Qayyarah becomes brighter at night than the city of Mosul ever was, as the oilfields are set on fire. Then, almost exactly when the blaze in Qayyarah is extinguished and the area darkens (i.e. when the blue line falls back to near zero), Mosul brightens once again (i.e. the red line rises) as the city is liberated.</p>
|
||||
|
||||
<!--
|
||||
### The Battle for Aleppo
|
||||
|
||||
The images below were taken between 2012 and 2014. Vast swaths of the city darken as neighbourhoods are razed by fighting.
|
||||
|
||||
The images below were taken between 2012 and 2014. Vast swaths of the city darken as neighborhoods are razed by fighting.
|
||||
|
||||
|
||||
<timelapse>
|
||||
|
||||
Though this is a trend that can be observed across the country, nowhere is the decline in nightlights more visible than in Aleppo. Below is a comparison of longitudinal trends in nighlights signatures between several cities:
|
||||
|
||||
Though this is a trend that can be observed across the country, nowhere is the decline in nightlights more visible than in Aleppo. Below is a comparison of longitudinal trends in nightlights signatures between several cities:
|
||||
|
||||
|
||||
<graph>
|
||||
|
||||
The most salient trend is Aleppo plummeting over the course of 2012, and becoming steadily darker over the course of the next four years. Raqqa drops in 2012 as well, but remains in flux until 2017, when the battle to reclaim the city pluges it into near total darkness. Damascus also experiences a dip in 2012, but stabilizes relatively quickly. The Turkish city of Gaziantep-- less than 100km from Aleppo and roughly 1/5th the size-- stands in stark contrast to the Syrian cities, becoming progressively brighter over the entire period.
|
||||
|
||||
The most salient trend is Aleppo plummeting over the course of 2012, and becoming steadily darker over the course of the next four years. Raqqa drops in 2012 as well, but remains in flux until 2017, when the battle to reclaim the city plunges it into near total darkness. Damascus also experiences a dip in 2012, but stabilizes relatively quickly. The Turkish city of Gaziantep -- less than 100km from Aleppo and roughly 1/5th the size -- stands in stark contrast to the Syrian cities, becoming progressively brighter over the entire period.
|
||||
|
||||
|
||||
Another interesting pattern here is the difference in seasonal trends in nightlights. Under normal circumstances in this part of the world, cities become brighter at night during the summer months. Restaurants, bars, and markets stay open later and conduct business outdoors. Gaziantep, which still attracts scores of tourists every year, displays pronounced seasonality. Damascus, the most stable of the three Syrian cities, also maintains a seasonal trend throughout the war. In contrast, both Raqqa and Aleppo maintain extremely low and roughly constant levels of nightlights year-round during the periods following intense fighting.
|
||||
|
||||
|
||||
Reliable economic data for Syria haven't been available for nearly a decade, and assessing the country's recovery is consequently difficult. But subtle indications of economic growth are visible above: all three Syrian cities have been on a steady upward trend since 2017, and beginning to display seasonal variation once again. -->
|
||||
<!-- ### Fighting for Oil
|
||||
|
||||
|
||||
Throughout the war, sudden massive spikes in nightlights signatures can be observed throughout the country. In the center of the map just west of Palmyra, some particularly large spikes occur in 2017:
|
||||
|
||||
These flashes of light show gas wells being set on fire, a common form of sabotage carried out by retreating Islamic State fighters. Modified Sentinel-2 imagery of the Hayyan gas field (indicated by the green box above) shows this in greater detail. Substituing the Red band in an RGB image with Near Infrared (NIR) highlights thermal signatures, showing fires burning brightly even during the day.
|
||||
|
||||
The large complex on the right is the Hayyan Gas Plant, which produced nearly 1/3 of Syria's electricity. The plant and its associated wells changed hands several times throughout the war, but were under Islamic State control until February 2017. In the video below, Islamic State fighters can be seen rigging the plant with explosives and destroying it on January 8th:
|
||||
These flashes of light show gas wells being set on fire, a common form of sabotage carried out by retreating Islamic State fighters. Modified Sentinel-2 imagery of the Hayyan gas field (indicated by the green box above) shows this in greater detail. Substituting the Red band in an RGB image with Near Infrared (NIR) highlights thermal signatures, showing fires burning brightly even during the day.
|
||||
|
||||
|
||||
The large complex on the right is the Hayyan Gas Plant, which produced nearly one third of Syria's electricity. The plant and its associated wells changed hands several times throughout the war, but were under Islamic State control until February 2017. In the video below, Islamic State fighters can be seen rigging the plant with explosives and destroying it on January 8th:
|
||||
|
||||
|
||||
In February, three Russian oil and gas companies (Zarubij Naft, Lukoil and Gazprom Neft) were given restoration, exploration and production rights to the hydrocarbon deposits West of Palmyra. On January 12th, 2017, the Syrian Army's 5th Legion and Russian special forces launched a counterattack known as the "Palmyra offensive", with the aim of retaking several important hydrocarbon deposits including Hayyan.
|
||||
|
||||
In February, three Russian oil and gas companies (Zarubij Naft, Lukoil and Gazprom Neft) were given restoration, exploration, and production rights to the hydrocarbon deposits West of Palmyra. On January 12th, 2017, the Syrian Army's 5th Legion and Russian special forces launched a counterattack known as the "Palmyra offensive", with the aim of retaking several important hydrocarbon deposits including Hayyan.
|
||||
|
||||
The timing of well fires aligns closely with a detailed timeline of the campaign.The Near Infrared Sentinel-2 image below shows the layout of the Hayyan Gas Plant and the wells in the Hayyan gas field:
|
||||
|
||||
|
||||
The Syrian Army took the Hayyan gas field on [February 4th](https://www.almasdarnews.com/article/syrian-army-liberates-hayyan-gas-fields-west-palmyra/), and retreating ISIS fighters set fire to wells 1, and 3. However, ISIS managed to briefly retake the Hayyan field on [February 7th](https://www.almasdarnews.com/article/isis-retakes-hayyan-gas-fields-new-bid-expand-west-palmyra/), setting fire to wells 2 and 4. These moments in the Palmyra Offensive are captured in NIR signatures
|
||||
|
||||
|
||||
Interestingly, despite the massive explosion caused by the bombing of the Hayyan Gas Plant, no prolonged thermal anomalies were detected over the area of the plant itself. The well fires, on the other hand, lasted for months. Below is an image of well fire at the Hayyan field taken from this [video](https://www.youtube.com/watch?v=WFe9abYyqK0); based on the nearby infrastructure and date (04/02/2017) of posting, it is likely Well-3.
|
||||
-->
|
||||
|
||||
@@ -808,13 +834,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./F5.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></span>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Vectors and Tables</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./refineries.html" class="pagination-link">
|
||||
<span class="nav-page-text">Refinery Identification</span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/C2_Refineries.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Refinery Identification</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
File diff suppressed because one or more lines are too long
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - Blast Damage Assessment</title>
|
||||
<title>Remote Sensing for OSINT - 9 Blast Damage Assessment</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -86,27 +86,27 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./object_detection.html" rel="next">
|
||||
<link href="./ships.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/C4_Ships.html" rel="next">
|
||||
<link href="../chapters/C2_Refineries.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -147,7 +147,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title">Blast Damage Assessment</h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Blast Damage Assessment</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -159,24 +159,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -219,17 +219,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -244,22 +244,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -274,27 +274,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link active">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -323,14 +323,14 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<li><a href="#further-research" id="toc-further-research" class="nav-link" data-scroll-target="#further-research">Further Research</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/blast.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/C3_Blast.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content page-columns page-full" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block">Blast Damage Assessment</h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Blast Damage Assessment</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -351,7 +351,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</div>
|
||||
<p>Assessing blast damage is a common task for open source investigators, and satellite imagery analysis is one of the best tools we have at our disposal to analyze this sort of phenomenon. <a href="https://earthobservatory.nasa.gov/images/147098/scientists-map-beirut-blast-damage">NASA</a> used Sentinel-1 imagery in the aftermath of the explosion to generate an estimated damage map. They explain that Sentinel-1 Synthetic Aperture Radar (SAR) imagery is good for this sort of task:</p>
|
||||
<p>“SAR instruments send pulses of microwaves toward Earth’s surface and listen for the reflections of those waves. The radar waves can penetrate cloud cover, vegetation, and the dark of night to detect changes that might not show up in visible light imagery. When Earth’s crust moves due to an earthquake, when dry land is suddenly covered by flood water, or when buildings have been damaged or toppled, the amplitude and phase of radar wave reflections changes in those areas and indicates to the satellite that something on the ground has changed.”</p>
|
||||
<p>The NASA team produced this estimate the day after the explosion, which is very impressive. However, due to the quick turnaround, were pretty light on the description of their methodology (they didnt provide any code, or even explicity say how exactly they generated the estimate), making their analysis hard to replicate. They also failed to validate their results, which is a critical step in any analysis.</p>
|
||||
<p>The NASA team produced this estimate the day after the explosion, which is very impressive. However, due to the quick turnaround, they were pretty light on the description of their methodology (they didn’t provide any code, or even explicitly say how exactly they generated the estimate), making their analysis hard to replicate. They also failed to validate their results, which is a critical step in any analysis.</p>
|
||||
<p>In this case study we’ll be developing our own change detection algorithm from scratch, applying to Sentinel-1 imagery of Beirut before and after the blast, and validating our results using building footprints and U.N. damage estimates as the ground truth. Below is the final result of the analysis, which shows building footprints colored according to the predicted level of damage they sustained from the blast:</p>
|
||||
<div class="column-page">
|
||||
<iframe src="https://ollielballinger.users.earthengine.app/view/beirutsar" width="100%" height="700px">
|
||||
@@ -378,9 +378,11 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p>Now lets go about implementing this in Earth Engine. We’ll start by centering the map on the port of Beirut, and setting the map to satellite view, and defining an area of interest (AOI) as a 3km buffer around the port:</p>
|
||||
<div class="sourceCode" id="cb1"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setCenter</span>(<span class="fl">35.51898</span><span class="op">,</span> <span class="fl">33.90153</span><span class="op">,</span> <span class="dv">15</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setOptions</span>(<span class="st">"satellite"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> aoi <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Point</span>(<span class="fl">35.51898</span><span class="op">,</span> <span class="fl">33.90153</span>)<span class="op">.</span><span class="fu">buffer</span>(<span class="dv">3000</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setOptions</span>(<span class="st">"satellite"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> aoi <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Point</span>(<span class="fl">35.51898</span><span class="op">,</span> <span class="fl">33.90153</span>)<span class="op">.</span><span class="fu">buffer</span>(<span class="dv">3000</span>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Next, let’s define a function in earth engine that will perform the T-Test. The block of code below defines a function to implement a t-test for every pixel in a set of images. The function will be called ‘ttest’, and takes four arguments:</p>
|
||||
<ul>
|
||||
<li>s1: the image collection</li>
|
||||
@@ -419,37 +421,42 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb2-28"><a href="#cb2-28" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">divide</span>(pre_n<span class="op">.</span><span class="fu">add</span>(post_n)<span class="op">.</span><span class="fu">subtract</span>(<span class="dv">2</span>))</span>
|
||||
<span id="cb2-29"><a href="#cb2-29" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">sqrt</span>()<span class="op">;</span></span>
|
||||
<span id="cb2-30"><a href="#cb2-30" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-31"><a href="#cb2-31" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the denominator of the t-test</span></span>
|
||||
<span id="cb2-32"><a href="#cb2-32" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> denom <span class="op">=</span> pooled_sd<span class="op">.</span><span class="fu">multiply</span>(</span>
|
||||
<span id="cb2-33"><a href="#cb2-33" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="fu">Number</span>(<span class="dv">1</span>)<span class="op">.</span><span class="fu">divide</span>(pre_n)<span class="op">.</span><span class="fu">add</span>(ee<span class="op">.</span><span class="fu">Number</span>(<span class="dv">1</span>)<span class="op">.</span><span class="fu">divide</span>(post_n))<span class="op">.</span><span class="fu">sqrt</span>()</span>
|
||||
<span id="cb2-34"><a href="#cb2-34" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb2-35"><a href="#cb2-35" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-36"><a href="#cb2-36" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the Degrees of Freedom, which is the number of observations minus 2</span></span>
|
||||
<span id="cb2-37"><a href="#cb2-37" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> df <span class="op">=</span> pre_n<span class="op">.</span><span class="fu">add</span>(post_n)<span class="op">.</span><span class="fu">subtract</span>(<span class="dv">2</span>)<span class="op">;</span></span>
|
||||
<span id="cb2-38"><a href="#cb2-38" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-39"><a href="#cb2-39" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"Number of Images: "</span><span class="op">,</span> df)<span class="op">;</span></span>
|
||||
<span id="cb2-31"><a href="#cb2-31" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-32"><a href="#cb2-32" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the denominator of the t-test</span></span>
|
||||
<span id="cb2-33"><a href="#cb2-33" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> denom <span class="op">=</span> pooled_sd<span class="op">.</span><span class="fu">multiply</span>(</span>
|
||||
<span id="cb2-34"><a href="#cb2-34" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="fu">Number</span>(<span class="dv">1</span>)<span class="op">.</span><span class="fu">divide</span>(pre_n)<span class="op">.</span><span class="fu">add</span>(ee<span class="op">.</span><span class="fu">Number</span>(<span class="dv">1</span>)<span class="op">.</span><span class="fu">divide</span>(post_n))<span class="op">.</span><span class="fu">sqrt</span>()</span>
|
||||
<span id="cb2-35"><a href="#cb2-35" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb2-36"><a href="#cb2-36" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-37"><a href="#cb2-37" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-38"><a href="#cb2-38" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the Degrees of Freedom, which is the number of observations minus 2</span></span>
|
||||
<span id="cb2-39"><a href="#cb2-39" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> df <span class="op">=</span> pre_n<span class="op">.</span><span class="fu">add</span>(post_n)<span class="op">.</span><span class="fu">subtract</span>(<span class="dv">2</span>)<span class="op">;</span></span>
|
||||
<span id="cb2-40"><a href="#cb2-40" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-41"><a href="#cb2-41" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the t-test using the:</span></span>
|
||||
<span id="cb2-42"><a href="#cb2-42" aria-hidden="true" tabindex="-1"></a> <span class="co">// mean of the pre-event period, </span></span>
|
||||
<span id="cb2-43"><a href="#cb2-43" aria-hidden="true" tabindex="-1"></a> <span class="co">// the mean of the post-event period, </span></span>
|
||||
<span id="cb2-44"><a href="#cb2-44" aria-hidden="true" tabindex="-1"></a> <span class="co">// and the pooled standard deviation</span></span>
|
||||
<span id="cb2-45"><a href="#cb2-45" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> change <span class="op">=</span> post_mean</span>
|
||||
<span id="cb2-46"><a href="#cb2-46" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">abs</span>()</span>
|
||||
<span id="cb2-47"><a href="#cb2-47" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">subtract</span>(pre_mean<span class="op">.</span><span class="fu">abs</span>())</span>
|
||||
<span id="cb2-48"><a href="#cb2-48" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">divide</span>(denom)</span>
|
||||
<span id="cb2-49"><a href="#cb2-49" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">abs</span>()</span>
|
||||
<span id="cb2-50"><a href="#cb2-50" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">subtract</span>(<span class="dv">2</span>)<span class="op">;</span></span>
|
||||
<span id="cb2-51"><a href="#cb2-51" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-52"><a href="#cb2-52" aria-hidden="true" tabindex="-1"></a> <span class="co">// return the t-values for each pixel</span></span>
|
||||
<span id="cb2-53"><a href="#cb2-53" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> change<span class="op">;</span></span>
|
||||
<span id="cb2-54"><a href="#cb2-54" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb2-41"><a href="#cb2-41" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-42"><a href="#cb2-42" aria-hidden="true" tabindex="-1"></a> <span class="fu">print</span>(<span class="st">"Number of Images: "</span><span class="op">,</span> df)<span class="op">;</span></span>
|
||||
<span id="cb2-43"><a href="#cb2-43" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-44"><a href="#cb2-44" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-45"><a href="#cb2-45" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the t-test using the:</span></span>
|
||||
<span id="cb2-46"><a href="#cb2-46" aria-hidden="true" tabindex="-1"></a> <span class="co">// mean of the pre-event period, </span></span>
|
||||
<span id="cb2-47"><a href="#cb2-47" aria-hidden="true" tabindex="-1"></a> <span class="co">// the mean of the post-event period, </span></span>
|
||||
<span id="cb2-48"><a href="#cb2-48" aria-hidden="true" tabindex="-1"></a> <span class="co">// and the pooled standard deviation</span></span>
|
||||
<span id="cb2-49"><a href="#cb2-49" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> change <span class="op">=</span> post_mean</span>
|
||||
<span id="cb2-50"><a href="#cb2-50" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">abs</span>()</span>
|
||||
<span id="cb2-51"><a href="#cb2-51" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">subtract</span>(pre_mean<span class="op">.</span><span class="fu">abs</span>())</span>
|
||||
<span id="cb2-52"><a href="#cb2-52" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">divide</span>(denom)</span>
|
||||
<span id="cb2-53"><a href="#cb2-53" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">abs</span>()</span>
|
||||
<span id="cb2-54"><a href="#cb2-54" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">subtract</span>(<span class="dv">2</span>)<span class="op">;</span></span>
|
||||
<span id="cb2-55"><a href="#cb2-55" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-56"><a href="#cb2-56" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-57"><a href="#cb2-57" aria-hidden="true" tabindex="-1"></a> <span class="co">// return the t-values for each pixel</span></span>
|
||||
<span id="cb2-58"><a href="#cb2-58" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> change<span class="op">;</span></span>
|
||||
<span id="cb2-59"><a href="#cb2-59" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>An important detail in the code above is that we’ve actually tweaked the t-test slightly, in two ways.</p>
|
||||
<p>First, the algorithm above returns tha <em>absolute</em> value of t (i.e. the absolute value of the difference between the two means). This is because we’re interested in whether the pixel has changed at all, not whether it’s changed in a particular direction. Second, we’ve subtracted 2 from the t-value.</p>
|
||||
<p>The t-value is a measure of how many standard deviations the difference between the two means is. Generally speaking, if the t-value is greater than 2, then the difference between the two means is considered statistically significant. 2 is a fairly abitrary cutoff, but it’s the most commonly used one since it corresponds to the 95% confidence interval (i.e., theres less than a 5% chance of observing a difference that big due to random chance). Now we’ve got a function that can take an image collection and return a t-value image, where a value greater than 0 corresponds to a statistically significant change between the pre-event and post-event periods.</p>
|
||||
<p>The t-value is a measure of how many standard deviations the difference between the two means is. Generally speaking, if the t-value is greater than 2, then the difference between the two means is considered statistically significant. 2 is a fairly arbitrary cutoff, but it’s the most commonly used one since it corresponds to the 95% confidence interval (i.e., there’s less than a 5% chance of observing a difference that big due to random chance). Now we’ve got a function that can take an image collection and return a t-value image, where a value greater than 0 corresponds to a statistically significant change between the pre-event and post-event periods.</p>
|
||||
</section>
|
||||
<section id="filtering-the-sentinel-1-imagery" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="filtering-the-sentinel-1-imagery">Filtering the Sentinel-1 Imagery</h2>
|
||||
<p>We can’t just blindly apply this algorithm to the entire image collection, because the image collection contains images from both ascending and descending orbits. We need to filter the image collection to the ascending and descending orbits, and then calculate the t-value for each orbit separately: this is because the satellite is viewing the scene from a completely different angle when it’s in ascending and descending orbits, which will generate a lot of noise in our data. In fact, even when the satellite is either ascending or descending, we can have multiple images of the same place taken from slightly different orbital tracks because these overlap (see <a href="./ch1#orbits">this visualization of orbits</a>). We need to filter the image collection to the relative orbit number that is most common within the image collection. For that, we define a new function called ‘filter_s1’, which takes a single argument: the path (either ‘ASCENDING’ or ‘DESCENDING’).</p>
|
||||
<p>We can’t just blindly apply this algorithm to the entire image collection, because the image collection contains images from both ascending and descending orbits. We need to filter the image collection to the ascending and descending orbits, and then calculate the t-value for each orbit separately: this is because the satellite is viewing the scene from a completely different angle when it’s in ascending and descending orbits, which will generate a lot of noise in our data. In fact, even when the satellite is either ascending or descending, we can have multiple images of the same place taken from slightly different orbital tracks because these overlap (see <a href="../ch1#orbits">this visualization of orbits</a>). We need to filter the image collection to the relative orbit number that is most common within the image collection. For that, we define a new function called ‘filter_s1’, which takes a single argument: the path (either ‘ASCENDING’ or ‘DESCENDING’).</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">filter_s1</span>(path) {</span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the image collection to the ascending or descending orbit</span></span>
|
||||
@@ -461,20 +468,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filterBounds</span>(aoi)</span>
|
||||
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">select</span>(<span class="st">"VH"</span>)<span class="op">;</span></span>
|
||||
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Find the most common relative orbit number</span></span>
|
||||
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> orbit <span class="op">=</span> s1</span>
|
||||
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">aggregate_array</span>(<span class="st">"relativeOrbitNumber_start"</span>)</span>
|
||||
<span id="cb3-15"><a href="#cb3-15" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">reduce</span>(ee<span class="op">.</span><span class="at">Reducer</span><span class="op">.</span><span class="fu">mode</span>())<span class="op">;</span></span>
|
||||
<span id="cb3-16"><a href="#cb3-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-17"><a href="#cb3-17" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the image collection to the most common relative orbit number</span></span>
|
||||
<span id="cb3-18"><a href="#cb3-18" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> s1 <span class="op">=</span> s1<span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">eq</span>(<span class="st">"relativeOrbitNumber_start"</span><span class="op">,</span> orbit))<span class="op">;</span></span>
|
||||
<span id="cb3-19"><a href="#cb3-19" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-20"><a href="#cb3-20" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the t-test for the filtered image collection using the function we defined earlier</span></span>
|
||||
<span id="cb3-21"><a href="#cb3-21" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> change <span class="op">=</span> <span class="fu">ttest</span>(s1<span class="op">,</span> <span class="st">"2020-08-04"</span><span class="op">,</span> <span class="dv">12</span><span class="op">,</span> <span class="dv">2</span>)<span class="op">;</span></span>
|
||||
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a> <span class="co">// Find the most common relative orbit number</span></span>
|
||||
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> orbit <span class="op">=</span> s1</span>
|
||||
<span id="cb3-15"><a href="#cb3-15" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">aggregate_array</span>(<span class="st">"relativeOrbitNumber_start"</span>)</span>
|
||||
<span id="cb3-16"><a href="#cb3-16" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">reduce</span>(ee<span class="op">.</span><span class="at">Reducer</span><span class="op">.</span><span class="fu">mode</span>())<span class="op">;</span></span>
|
||||
<span id="cb3-17"><a href="#cb3-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-18"><a href="#cb3-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-19"><a href="#cb3-19" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the image collection to the most common relative orbit number</span></span>
|
||||
<span id="cb3-20"><a href="#cb3-20" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> s1 <span class="op">=</span> s1<span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">eq</span>(<span class="st">"relativeOrbitNumber_start"</span><span class="op">,</span> orbit))<span class="op">;</span></span>
|
||||
<span id="cb3-21"><a href="#cb3-21" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-22"><a href="#cb3-22" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-23"><a href="#cb3-23" aria-hidden="true" tabindex="-1"></a> <span class="co">// Return the t-values for each pixel</span></span>
|
||||
<span id="cb3-24"><a href="#cb3-24" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> change<span class="op">;</span></span>
|
||||
<span id="cb3-25"><a href="#cb3-25" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb3-23"><a href="#cb3-23" aria-hidden="true" tabindex="-1"></a> <span class="co">// Calculate the t-test for the filtered image collection using the function we defined earlier</span></span>
|
||||
<span id="cb3-24"><a href="#cb3-24" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> change <span class="op">=</span> <span class="fu">ttest</span>(s1<span class="op">,</span> <span class="st">"2020-08-04"</span><span class="op">,</span> <span class="dv">12</span><span class="op">,</span> <span class="dv">2</span>)<span class="op">;</span></span>
|
||||
<span id="cb3-25"><a href="#cb3-25" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-26"><a href="#cb3-26" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-27"><a href="#cb3-27" aria-hidden="true" tabindex="-1"></a> <span class="co">// Return the t-values for each pixel</span></span>
|
||||
<span id="cb3-28"><a href="#cb3-28" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> change<span class="op">;</span></span>
|
||||
<span id="cb3-29"><a href="#cb3-29" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>You’ll notice that we’ve called the ttest function we defined earlier with four arguments:</p>
|
||||
<ul>
|
||||
<li>s1: the Sentinel-1 image collection filtered to the ascending or descending orbit</li>
|
||||
@@ -484,61 +495,67 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</ul>
|
||||
<p>Now we want to apply this function to the image collection twice (once for each orbit) and then combine the two images into a single image. After that, we can clip it to the area of interest and display it on the map:</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a><span class="co">// Call the filter_s1 function twice, once for each orbit, and then combine the two images into a single image</span></span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> composite <span class="op">=</span> ee</span>
|
||||
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">ImageCollection</span>([<span class="fu">filter_s1</span>(<span class="st">"ASCENDING"</span>)<span class="op">,</span> <span class="fu">filter_s1</span>(<span class="st">"DESCENDING"</span>)])</span>
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">mean</span>()</span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">clip</span>(aoi)<span class="op">;</span></span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a><span class="co">// Define a color palette</span></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> palette <span class="op">=</span> [<span class="st">"440154"</span><span class="op">,</span> <span class="st">"3b528b"</span><span class="op">,</span> <span class="st">"21918c"</span><span class="op">,</span> <span class="st">"5ec962"</span><span class="op">,</span> <span class="st">"fde725"</span>]<span class="op">;</span></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the composite to the map</span></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(</span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a> composite<span class="op">,</span></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> { <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="dv">4</span><span class="op">,</span> <span class="dt">opacity</span><span class="op">:</span> <span class="fl">0.8</span><span class="op">,</span> <span class="dt">palette</span><span class="op">:</span> palette }<span class="op">,</span></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> <span class="st">"change"</span></span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The visualization parameters correspond the statitical significance of the change in pixel values. Using the Viridis color palette which ranges from purple to yellow, dark purple pixels indicate no significant change, and yellow pixels indicate a significant change with with 95% confidence. The brighter the yellow of a pixel, the more significant the change.</p>
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a><span class="co">// Call the filter_s1 function twice, once for each orbit, and then combine the two images into a single image</span></span>
|
||||
<span id="cb4-4"><a href="#cb4-4" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> composite <span class="op">=</span> ee</span>
|
||||
<span id="cb4-5"><a href="#cb4-5" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">ImageCollection</span>([<span class="fu">filter_s1</span>(<span class="st">"ASCENDING"</span>)<span class="op">,</span> <span class="fu">filter_s1</span>(<span class="st">"DESCENDING"</span>)])</span>
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">mean</span>()</span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">clip</span>(aoi)<span class="op">;</span></span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a><span class="co">// Define a color palette</span></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> palette <span class="op">=</span> [<span class="st">"440154"</span><span class="op">,</span> <span class="st">"3b528b"</span><span class="op">,</span> <span class="st">"21918c"</span><span class="op">,</span> <span class="st">"5ec962"</span><span class="op">,</span> <span class="st">"fde725"</span>]<span class="op">;</span></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the composite to the map</span></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(</span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a> composite<span class="op">,</span></span>
|
||||
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a> { <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="dv">4</span><span class="op">,</span> <span class="dt">opacity</span><span class="op">:</span> <span class="fl">0.8</span><span class="op">,</span> <span class="dt">palette</span><span class="op">:</span> palette }<span class="op">,</span></span>
|
||||
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a> <span class="st">"change"</span></span>
|
||||
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The visualization parameters correspond to the statistical significance of the change in pixel values. Using the Viridis color palette which ranges from purple to yellow, dark purple pixels indicate no significant change, and yellow pixels indicate a significant change with with 95% confidence. The brighter the yellow of a pixel, the more significant the change.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/beirut/beirut_change_2020.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/beirut/beirut_change_2020.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Pixelwise T-Test, 2020</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>This seems to be working quite well; but remember, ports are generally prone to change. The t-test is accounting for this by calculating each pixel’s variance over the entire time period, but it’s still possible that the change we’re seeing is due to the port rather than the explosion. To test this, we can run the same algorithm on the same area, using the same date cutoff (August 4th), but in a different year; i’ve chosen 2018. This is what’s known as a placebo test: if it’s still showing loads of statistically significant change around the cutoff, our algorithm is probably picking up on port activity rather than the explosion.</p>
|
||||
<p>This seems to be working quite well; but remember, ports are generally prone to change. The t-test is accounting for this by calculating each pixel’s variance over the entire time period, but it’s still possible that the change we’re seeing is due to the port rather than the explosion. To test this, we can run the same algorithm on the same area, using the same date cutoff (August 4th), but in a different year; I’ve chosen 2018. This is what’s known as a placebo test: if it’s still showing loads of statistically significant change around the cutoff, our algorithm is probably picking up on port activity rather than the explosion.</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/beirut/beirut_change_2018.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/beirut/beirut_change_2018.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Pixelwise T-Test, 2018</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Compared to the 2020 image, there’s a lot less yellow (significant change). That being said there are a few yellow areas. This could be due to a number of reasons: ships coming and going, cranes moving, and containers being loaded and unloaded would all register in the change detection algorithm. There are also a number of yellow specks throughout the city, which is also to be expected since cities are also generally in a state of flux. Construction, demolition, and even the growth of vegetation can all be detected by the algorithm.</p>
|
||||
<p>However, the scale and quantity of the change is nowhere near what it was for the 2020 image. This is a good sign that the algorithm detecting change resulting from the explosion.</p>
|
||||
<p>Compared to the 2020 image, there’s a lot less yellow (significant change). That being said, there are a few yellow areas. This could be due to a number of reasons: ships coming and going, cranes moving, and containers being loaded and unloaded would all register in the change detection algorithm. There are also a number of yellow specks throughout the city, which is also to be expected since cities are also generally in a state of flux. Construction, demolition, and even the growth of vegetation can all be detected by the algorithm.</p>
|
||||
<p>However, the scale and quantity of the change is nowhere near what it was for the 2020 image. This is a good sign that the algorithm is detecting change resulting from the explosion.</p>
|
||||
</section>
|
||||
<section id="validation" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="validation">Validation</h2>
|
||||
<p>Great. We’ve developed our very own change detection algorithm in earth engine, applied it to the Beirut explosion, and it seems to be working using a basic placebo test. But how do we know that it’s correctly predicting the <em>extent</em> of the damage, and not wildly over/underestimating?</p>
|
||||
<p>Given that this was a few years ago, we have the benefit of hindsight. In particular, the United Nations and the Municipality of Beirut have <a href="https://unhabitat.org/sites/default/files/2020/10/municipality_of_beirut_-_beirut_explosion_rapid_assessment_report.pdf">published a report</a> on the damage caused by the explosion. This report includes estimates of the number of buildings damaged or destroyed by the explosion, as well as the number of people displaced. The report states that approximately 10,000 buildings were damaged within a 3km radius of the port. If our algorithm suggests that only 1,000 buildings were damaged, it’s undershooting. If it suggests that 100,000 buildings were damaged, it’s overshooting.</p>
|
||||
<p>Using building footprint data and the t-test image we just generated, we can generate an estimate of the number of damaged buildings according to our model. First, we want to generate a thresholded image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0. We can then use this mask to reduce the building footprints to a single value for each building, where the value is the mean of the t-test image within the footprint. If the mean value is greater than 0, the building is damaged. If it’s less than 0, the building is not damaged.</p>
|
||||
<p>Great. We’ve developed our very own change detection algorithm in earth engine, applied it to the Beirut explosion, and it seems to be working after checking with a basic placebo test. But how do we know that it’s correctly predicting the <em>extent</em> of the damage, and not wildly over/underestimating?</p>
|
||||
<p>Given that this was a few years ago, we have the benefit of hindsight. In particular, the United Nations and the Municipality of Beirut have <a href="https://unhabitat.org/sites/default/files/2020/10/municipality_of_beirut_-_beirut_explosion_rapid_assessment_report.pdf">published a report</a> on the damage caused by the explosion. This report includes estimates of the number of buildings damaged or destroyed, as well as the number of people displaced. The report states that approximately 10,000 buildings were damaged within a 3 kilometre radius of the port. If our algorithm suggests that only 1,000 buildings were damaged, it’s undershooting. If it suggests that 100,000 buildings were damaged, it’s overshooting.</p>
|
||||
<p>Using building footprint data and the t-test image we just generated, we can createe an estimate of the number of damaged buildings according to our model. First, we want to generate a thresholded image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0. We can then use this mask to reduce the building footprints to a single value for each building, where the value is the mean of the t-test image within the footprint. If the mean value is greater than 0, the building is damaged. If it’s less than 0, the building is not damaged.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a mask of the t-test image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0</span></span>
|
||||
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> threshold <span class="op">=</span> composite<span class="op">.</span><span class="fu">updateMask</span>(composite<span class="op">.</span><span class="fu">gt</span>(<span class="dv">0</span>))<span class="op">;</span></span>
|
||||
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a><span class="co">// Load the building footprints</span></span>
|
||||
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> buildings <span class="op">=</span> ee</span>
|
||||
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">FeatureCollection</span>(<span class="st">"projects/sat-io/open-datasets/MSBuildings/Lebanon"</span>)</span>
|
||||
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filterBounds</span>(aoi)<span class="op">;</span></span>
|
||||
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a><span class="co">// Calculate the mean value of the t-test image within each building footprint</span></span>
|
||||
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> damaged_buildings <span class="op">=</span> threshold<span class="op">.</span><span class="fu">reduceRegions</span>({</span>
|
||||
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> buildings<span class="op">,</span></span>
|
||||
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">reducer</span><span class="op">:</span> ee<span class="op">.</span><span class="at">Reducer</span><span class="op">.</span><span class="fu">mean</span>()<span class="op">,</span></span>
|
||||
<span id="cb5-13"><a href="#cb5-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb5-14"><a href="#cb5-14" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb5-15"><a href="#cb5-15" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-16"><a href="#cb5-16" aria-hidden="true" tabindex="-1"></a><span class="co">// Print the number of buildings with a mean value greater than 0</span></span>
|
||||
<span id="cb5-17"><a href="#cb5-17" aria-hidden="true" tabindex="-1"></a><span class="co">// i.e., those displaying statistically significant change</span></span>
|
||||
<span id="cb5-18"><a href="#cb5-18" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(damaged_buildings<span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">gt</span>(<span class="st">"mean"</span><span class="op">,</span> <span class="dv">0</span>))<span class="op">.</span><span class="fu">size</span>())<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a><span class="co">// Load the building footprints</span></span>
|
||||
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> buildings <span class="op">=</span> ee</span>
|
||||
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">FeatureCollection</span>(<span class="st">"projects/sat-io/open-datasets/MSBuildings/Lebanon"</span>)</span>
|
||||
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filterBounds</span>(aoi)<span class="op">;</span></span>
|
||||
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a><span class="co">// Calculate the mean value of the t-test image within each building footprint</span></span>
|
||||
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> damaged_buildings <span class="op">=</span> threshold<span class="op">.</span><span class="fu">reduceRegions</span>({</span>
|
||||
<span id="cb5-13"><a href="#cb5-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">collection</span><span class="op">:</span> buildings<span class="op">,</span></span>
|
||||
<span id="cb5-14"><a href="#cb5-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">reducer</span><span class="op">:</span> ee<span class="op">.</span><span class="at">Reducer</span><span class="op">.</span><span class="fu">mean</span>()<span class="op">,</span></span>
|
||||
<span id="cb5-15"><a href="#cb5-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb5-16"><a href="#cb5-16" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb5-17"><a href="#cb5-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-18"><a href="#cb5-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-19"><a href="#cb5-19" aria-hidden="true" tabindex="-1"></a><span class="co">// Print the number of buildings with a mean value greater than 0</span></span>
|
||||
<span id="cb5-20"><a href="#cb5-20" aria-hidden="true" tabindex="-1"></a><span class="co">// i.e., those displaying statistically significant change</span></span>
|
||||
<span id="cb5-21"><a href="#cb5-21" aria-hidden="true" tabindex="-1"></a><span class="fu">print</span>(damaged_buildings<span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">gt</span>(<span class="st">"mean"</span><span class="op">,</span> <span class="dv">0</span>))<span class="op">.</span><span class="fu">size</span>())<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The result is 9,256, which is pretty damn close to 10,000. We can also visualize the building footprints on the map, colored according the mean value of the t-test image within the footprint, where:</p>
|
||||
<ul>
|
||||
<li>Blue = no damage</li>
|
||||
@@ -547,62 +564,66 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<li>Red = high levels of damage</li>
|
||||
</ul>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a><span class="co">// Create an empty image</span></span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> empty <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>()<span class="op">.</span><span class="fu">byte</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a><span class="co">// Paint the building footprints onto the empty image</span></span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> outline <span class="op">=</span> empty<span class="op">.</span><span class="fu">paint</span>({</span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">featureCollection</span><span class="op">:</span> damaged_buildings<span class="op">,</span></span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">color</span><span class="op">:</span> <span class="st">"mean"</span><span class="op">,</span></span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">width</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a><span class="co">// Define a color palette</span></span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> building_palette <span class="op">=</span> [</span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> <span class="st">"0034f5"</span><span class="op">,</span></span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> <span class="st">"1e7d83"</span><span class="op">,</span></span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> <span class="st">"4da910"</span><span class="op">,</span></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a> <span class="st">"b3c120"</span><span class="op">,</span></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> <span class="st">"fcc228"</span><span class="op">,</span></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> <span class="st">"ff8410"</span><span class="op">,</span></span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> <span class="st">"fd3000"</span><span class="op">,</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a>]<span class="op">;</span></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the image to the map</span></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(</span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a> outline<span class="op">,</span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a> { <span class="dt">palette</span><span class="op">:</span> building_palette<span class="op">,</span> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="dv">2</span> }<span class="op">,</span></span>
|
||||
<span id="cb6-27"><a href="#cb6-27" aria-hidden="true" tabindex="-1"></a> <span class="st">"Damaged Buildings"</span></span>
|
||||
<span id="cb6-28"><a href="#cb6-28" aria-hidden="true" tabindex="-1"></a>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The result naturally resembles the underlying t-test image, with high levels of damage concetrated around the port, and progressively decreasing damage with distance:</p>
|
||||
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a><span class="co">// Create an empty image</span></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> empty <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>()<span class="op">.</span><span class="fu">byte</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a><span class="co">// Paint the building footprints onto the empty image</span></span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> outline <span class="op">=</span> empty<span class="op">.</span><span class="fu">paint</span>({</span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">featureCollection</span><span class="op">:</span> damaged_buildings<span class="op">,</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">color</span><span class="op">:</span> <span class="st">"mean"</span><span class="op">,</span></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">width</span><span class="op">:</span> <span class="dv">5</span><span class="op">,</span></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a><span class="co">// Define a color palette</span></span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> building_palette <span class="op">=</span> [</span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a> <span class="st">"0034f5"</span><span class="op">,</span></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> <span class="st">"1e7d83"</span><span class="op">,</span></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> <span class="st">"4da910"</span><span class="op">,</span></span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> <span class="st">"b3c120"</span><span class="op">,</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> <span class="st">"fcc228"</span><span class="op">,</span></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a> <span class="st">"ff8410"</span><span class="op">,</span></span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a> <span class="st">"fd3000"</span><span class="op">,</span></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a>]<span class="op">;</span></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-27"><a href="#cb6-27" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the image to the map</span></span>
|
||||
<span id="cb6-28"><a href="#cb6-28" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">addLayer</span>(</span>
|
||||
<span id="cb6-29"><a href="#cb6-29" aria-hidden="true" tabindex="-1"></a> outline<span class="op">,</span></span>
|
||||
<span id="cb6-30"><a href="#cb6-30" aria-hidden="true" tabindex="-1"></a> { <span class="dt">palette</span><span class="op">:</span> building_palette<span class="op">,</span> <span class="dt">min</span><span class="op">:</span> <span class="dv">0</span><span class="op">,</span> <span class="dt">max</span><span class="op">:</span> <span class="dv">2</span> }<span class="op">,</span></span>
|
||||
<span id="cb6-31"><a href="#cb6-31" aria-hidden="true" tabindex="-1"></a> <span class="st">"Damaged Buildings"</span></span>
|
||||
<span id="cb6-32"><a href="#cb6-32" aria-hidden="true" tabindex="-1"></a>)<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The result naturally resembles the underlying t-test image, with high levels of damage concentrated around the port, and progressively decreasing damage with distance:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/beirut/beirut_footprints.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/beirut/beirut_footprints.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Building Footprints colored according to estimated blast damage</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>To get a better sense of how these predicitons correspond to actual damage, we can zoom in and turn on the Google satellite basemap, which has imagery taken just after the explosion; you can still see capsized boats in the port. Zooming in to the epicentre, we can see several warehouses that were effectively vaporized. Our change detection algorithm picks up on a high degree of change, as indicated by the red outlines of the building footprints:</p>
|
||||
<p>To get a better sense of how these predicitions correspond to actual damage, we can zoom in and turn on the Google satellite basemap, which has imagery taken just after the explosion; you can still see capsized boats in the port. Zooming in to the epicenter, we can see several warehouses that were effectively vaporized. Our change detection algorithm picks up on a high degree of change, as indicated by the red outlines of the building footprints:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/beirut/beirut_footprints_port.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/beirut/beirut_footprints_port.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Predicted damage and optical satellite imagery in the Port of Beirut, August 2020</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>This is pretty low-hanging fruit. Let’s look at a different area, around 1.3km east from the epicentre with a mix of warehouses and residential buildings:</p>
|
||||
<p>This is pretty low-hanging fruit. Let’s look at a different area, around 1.3km east from the epicenter with a mix of warehouses and residential buildings:</p>
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
<figure class="figure">
|
||||
<p><img src="./images/beirut/beirut_footprints_zoomed.jpg" class="img-fluid figure-img"></p>
|
||||
<p><img src="../images/beirut/beirut_footprints_zoomed.jpg" class="img-fluid figure-img"></p>
|
||||
<p></p><figcaption class="figure-caption">Area east of the port: 35.533194, 33.9024</figcaption><p></p>
|
||||
</figure>
|
||||
</div>
|
||||
<p>Here, there’s greater variation in the predictions. I’ve highlighted three areas.</p>
|
||||
<p>In Area A, we see a warehouse with a highily deformed roof; panels of corrugated iron are missing, and much of the roof is warped. The building footprint for this warehouse is red, suggesting that our algorithm correctly predicts a significant amount of blast damage.</p>
|
||||
<p>In Area B, we see a medium-rise building. If you look closely at the southern edge of the building, you’ll see the siding has been completely torn off and is laying on the sidewalk. The bulding footprint is orange, suggesting a medium amount of change. We may be underestimating a bit here.</p>
|
||||
<p>In Area A, we see a warehouse with a highly deformed roof; panels of corrugated iron are missing, and much of the roof is warped. The building footprint for this warehouse is red, suggesting that our algorithm correctly predicts a significant amount of blast damage.</p>
|
||||
<p>In Area B, we see a medium-rise building. If you look closely at the southern edge of the building, you’ll see the siding has been completely torn off and is laying on the sidewalk. The building footprint is orange, suggesting a medium amount of change. We may be underestimating a bit here.</p>
|
||||
<p>In Area C, there are a bunch of high rise buildings clustered together. The building footprints are all blue, suggesting little to no damage. This is a bit of a surprise given how damaged areas A and B are. If you squint at the satellite image, it is indeed hard to tell if these buildings are damaged because we’re looking at them from the top down, when much of the damage (e.g., the windows being blown out) would only be visible from the side. Indeed, our own estimate of the number of damaged buildings based on the algorithm we developed is about 8% shy of the U.N.’s estimate. This may be why.</p>
|
||||
</section>
|
||||
<section id="conclusion" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="conclusion">Conclusion</h2>
|
||||
<p>In this practical, we created a custom change detection algorithm that conducts a pixelwise t-test to detect change resulting from the 2020 explosion in the port of Beirut. By defining our own functions to do most of this analysis, we can apply the same workflow quite easily to a different context by simply moving the AOI and inputting the date of the shock. A placebo test showed that it’s not just detecting general change in the area, but specifically change resulting from the explosion: when we keep everythgin the same but change the year of the shock, we see very little significant change being detected. Finally, by joining the predicted damage map to building footprints, we come up with an estimate of 9,256 damaged buildings, which is pretty close to the U.N.’s estimate of 10,000. That concludes the portion of this case study that deals with Earth Engine, but if you’re interested in learning more about why we’re coming up a bit short on the damage estimate (and some different ways of looking at the problem), read on.</p>
|
||||
<p>In this practical example, we created a custom change detection algorithm that conducts a pixelwise t-test to detect change resulting from the 2020 explosion in the port of Beirut. By defining our own functions to do most of this analysis, we can apply the same workflow quite easily to a different context by simply moving the AOI and inputting the date of the shock. A placebo test showed that it’s not just detecting general change in the area, but specifically change resulting from the explosion: when we keep everything the same but change the year of the shock, we see very little significant change being detected. Finally, by joining the predicted damage map to building footprints, we come up with an estimate of 9,256 damaged buildings, which is pretty close to the U.N.’s estimate of 10,000. That concludes the portion of this case study that deals with Earth Engine, but if you’re interested in learning more about why we’re coming up a bit short on the damage estimate (and some different ways of looking at the problem), read on.</p>
|
||||
</section>
|
||||
<section id="extension-satellite-imagery-and-its-limits" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="extension-satellite-imagery-and-its-limits">Extension: Satellite Imagery and its Limits</h2>
|
||||
@@ -612,88 +633,87 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p lang="en" dir="ltr">
|
||||
Stunning video shows explosions just minutes ago at Beirut port <a href="https://t.co/ZjltF0VcTr">pic.twitter.com/ZjltF0VcTr</a>
|
||||
</p>
|
||||
— Borzou Daragahi 🖊🗒 (<span class="citation" data-cites="borzou">(<a href="#ref-borzou" role="doc-biblioref"><strong>borzou?</strong></a>)</span>) <a href="https://twitter.com/borzou/status/1290675854767513600?ref_src=twsrc%5Etfw">August 4, 2020</a>
|
||||
— Borzou Daragahi 🖊🗒 (<span class="citation" data-cites="borzou">@borzou</span>) <a href="https://twitter.com/borzou/status/1290675854767513600?ref_src=twsrc%5Etfw">August 4, 2020</a>
|
||||
</blockquote>
|
||||
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
|
||||
<p>Geolocating this video was pretty simple thanks to the Greek Orthodox church (highlighted in green below) and the road leading to it (highlighted in blue). The red box indicates the likely location (33.889061, 35.515909) from which the person was filming:</p>
|
||||
<p><img src="./images/beirut/IMG_2.png" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/IMG_2.png" class="img-fluid"></p>
|
||||
<p>The video shows heavy damage being sustained by areas well outside the zones classified as damaged in the maps above (both my own and NASA’s). Indeed, substantial damage was reported several kilometers away.</p>
|
||||
<p>Why are satellite images underestimating damage in Beirut? Satellite images are taken from above, and are two-dimensional. Much of the damage caused by the blast, however, was directional; the pressure wave hit the sides of buildings, as shown in this diagram from a FEMA manual:</p>
|
||||
<p><img src="./images/beirut/IMG_3.png" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/IMG_3.png" class="img-fluid"></p>
|
||||
<p>Areas close to the explosion suffered so much damage that it could be seen from above, but even if an apartment building had all of its windows blown out, this would not necessarily be visible in a top-down view. Even for radar, which does technically collect data in three dimensions, the angle problem remains; a high resolution radar might be able to tell you how tall an apartment complex is, but it won’t give you a clear image of all sides. Case in point: the NASA damage map was created using Sentinel-1 SAR data. In a nutshell, damage assessment in this case is a three-dimensional problem, and remote sensing is a two-dimensional solution.</p>
|
||||
<section id="creating-a-3d-model-of-beirut" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="creating-a-3d-model-of-beirut">Creating a 3D model of Beirut</h3>
|
||||
<p>To create a more accurate rendering of directional blast damage, three dimensional data are required. Data from Open Street Maps (OSM) contains information on both the “footprints” (i.e., the location and shape) as well as the height of buildings, which is enough to create a three dimensional model of Beirut. 3D rendering was done in Blender using the Blender-OSM add-on to import a satellite basemap, terrain raster, and OSM data.</p>
|
||||
<p>Geolocated videos of the blast can be used to verify and adjust the model. Below is a side-by-side comparison of the twitter video and a 3D rendition of OSM data:</p>
|
||||
<p><img src="./images/beirut/IMG_4.png" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/IMG_4.png" class="img-fluid"></p>
|
||||
<p>Some slight adjustments to the raw OSM data were made to achieve the image on the right. The building footprints are generally very accurate and comprehensive in coverage, but the building height data does occasionally have to be adjusted manually. A simple and reliable way of doing this is to look at the shadows cast by the building on the satellite base map and scale accordingly. I also added a rough texture to the buildings to help differentiate them, and added the domed roof of the Greek Orthodox church for reference.</p>
|
||||
<p>For good measure, a second video is geolocated following the same procedure:</p>
|
||||
<blockquote class="twitter-tweet blockquote">
|
||||
<p lang="en" dir="ltr">
|
||||
Another view of the explosions in Beirut <a href="https://t.co/efT5VlpMkj">pic.twitter.com/efT5VlpMkj</a>
|
||||
</p>
|
||||
— Borzou Daragahi 🖊🗒 (<span class="citation" data-cites="borzou">(<a href="#ref-borzou" role="doc-biblioref"><strong>borzou?</strong></a>)</span>) <a href="https://twitter.com/borzou/status/1290678580897251330?ref_src=twsrc%5Etfw">August 4, 2020</a>
|
||||
— Borzou Daragahi 🖊🗒 (<span class="citation" data-cites="borzou">@borzou</span>) <a href="https://twitter.com/borzou/status/1290678580897251330?ref_src=twsrc%5Etfw">August 4, 2020</a>
|
||||
</blockquote>
|
||||
<script async="" src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
|
||||
<p>The second pier (highlighted in green) and the angle (in blue) serve as references:</p>
|
||||
<p><img src="./images/beirut/IMG_5.png" class="img-fluid"></p>
|
||||
<p>The video was taken from the rooftop of a japanese restaurant called Clap Beirut (in red above). This is confirmed by a picture of the rooftop bar on google images, which matches the bar that can be seen at 0:02 in the twitter video. Below is a comparison of the video view and the 3D OSM model:</p>
|
||||
<p><img src="./images/beirut/IMG_6.png" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/IMG_5.png" class="img-fluid"></p>
|
||||
<p>The video was taken from the rooftop of a Japanese restaurant called Clap Beirut (in red above). This is confirmed by a picture of the rooftop bar on google images, which matches the bar that can be seen at 0:02 in the twitter video. Below is a comparison of the video view and the 3D OSM model:</p>
|
||||
<p><img src="../images/beirut/IMG_6.png" class="img-fluid"></p>
|
||||
<p>Though somewhat grainy, the basemap on the OSM rendering shows the same parking lot in the foreground, the second pier, and the same two buildings highlighted in yellow. Having created a 3D model of Beirut using OSM data, we can now simulate how the explosion would interact with the cityscape.</p>
|
||||
</section>
|
||||
<section id="using-a-viewshed-analysis-to-assess-blast-exposure" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="using-a-viewshed-analysis-to-assess-blast-exposure">Using a Viewshed Analysis to Assess Blast Exposure</h3>
|
||||
<p>As the pressure wave moved through the Beirut, some buildings bore the full force of the explosion, while others were partially shielded by taller structures. A viewshed analysis can be conducted to identify surfaces that were directly exposed to the explosion by creating a lighting object at ground zero; areas that are lit up experienced unobstructed exposure to the blast:</p>
|
||||
<p><img src="./images/beirut/GIF_1.gif" class="img-fluid"></p>
|
||||
<p>Pressure waves, like sound, are capable of diffraction (beding around small obstructions). To roughly simluate this, the lighting object is gradually raised, allowing the light to pass “around” obstructions. Warehouses on the Eastern side of the docks, as well as the first row of apartment buildings facing the docks are immediately affected. As the lighting object rises above the warehouse, more areas suffer direct exposure.</p>
|
||||
<p><img src="../images/beirut/GIF_1.gif" class="img-fluid"></p>
|
||||
<p>Pressure waves, like sound, are capable of diffraction (bending around small obstructions). To roughly simulate this, the lighting object is gradually raised, allowing the light to pass “around” obstructions. Warehouses on the Eastern side of the docks, as well as the first row of apartment buildings facing the docks are immediately affected. As the lighting object rises above the warehouse, more areas suffer direct exposure.</p>
|
||||
<p>Using two lighting objects– a red one at 10 meters and a blue one at 20 meters above the warehouse at ground zero– the intensity of the blast in different areas is highlighted; red areas suffered direct exposure, blue areas suffered partially obstructed exposure, and black areas were indirectly exposed.</p>
|
||||
<p><img src="./images/beirut/IMG_7.png" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/IMG_7.png" class="img-fluid"></p>
|
||||
<p>In the immediate vicinity of the explosion the large “L” shaped building (Lebanon’s strategic grain reserve) is bright red, and was barely left standing. It absorbed a large amount of the blast, shielding areas behind it and thereby casting a long blue shadow to the West. If one refers back to the satellite damage maps above, there appears to be significantly less damage in the area just West of (“behind”) the grain silo, roughly corresponding to the blue shadow above. While these areas were still heavily damaged, they seem to have suffered less damage than areas of equal distance to the East.</p>
|
||||
</section>
|
||||
<section id="accounting-for-diffraction" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="accounting-for-diffraction">Accounting for Diffraction</h3>
|
||||
<p>The viewshed analysis tells us which sides of a building are exposed to the blast, but it’s a pretty rough approximation of the way the pressure wave would respond to obstacles in its path. As previously mentioned, pressure waves behave much like sound waves or waves in water: they bounce off of objects, move around obstructions, and gradually fade.</p>
|
||||
<p>To get a more precise idea of the way in which the blast interacted with the urban environment, we can model the blast as an actual wave using the “dynamic wave” feature in Blender. This effectively involves creating a two-dimensional plane, telling it to behave like water, and simulating an object being dropped into the water. By putting an obstruction in this plane, we can see how the wave responds to it. As an example, the grain silo has been isolated below:</p>
|
||||
<p><img src="./images/beirut/GIF_2.gif" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/GIF_2.gif" class="img-fluid"></p>
|
||||
<p>As the blast hits the side of the silo, it is reflected. Two large waves can be seen traveling to the right: the initial blast wave, and the reflection from the silo which rivals the initial wave in magnitude. To the left, the wave travels around the silo but is significantly weakened.</p>
|
||||
<p>Broadening the focus and adding the rest of the OSM data back in, we can observe how the pressure wave interacted with buildings on the waterfront:</p>
|
||||
<p><img src="./images/beirut/GIF_3.gif" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/GIF_3.gif" class="img-fluid"></p>
|
||||
<p>The warehouses on the docks were omitted to emphasize the interaction between the pressure wave and the waterfront buildings; their light metal structure and low height means they would have caused little reflection anyway. The general pattern of the dynamic wave is consistent with the viewshed, but adds a layer of detail. The blast is reflected off of the silo towards the East, leading to a double hit. Though the wave still moves around the silo to the West, the pressure is diminished. Once the wave hits the highrises, the pattern becomes noisy as the wave both presses forward into the mainland and is reflected back towards the pier.</p>
|
||||
</section>
|
||||
<section id="modeling-the-pressure-wave" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="modeling-the-pressure-wave">Modeling the Pressure Wave</h3>
|
||||
<p>Now that we’ve accounted for the directionality of the blast and the influence of buildings, we can model the pressure wave itself. An expanding sphere centered at ground zero is used to model the progression of the pressure wave through the city. To get a visual sense of the blast’s force, the color of the sphere will be a function of the pressure exerted by pressure wave.</p>
|
||||
<p>The pressure exerted by the explosion in kilopascals (kPa) at various distances can be calculated using the DoD’s Blast Effects Computer, which allows users to input variables such as the TNT equivalent of the ordnance, storage method, and elevation. Though there are several estimates, the blast was likely equivalent to around 300 tons of TNT. The direct “incident pressure” of the pressure wave is shown in blue. However, pressure waves from explosions that occur on the ground are reflected upwards, amplifying the total pressure exerted by the blast. This “reflected pressure” is shown in orange:</p>
|
||||
<p>Now that we’ve accounted for the directionality of the blast and the influence of buildings, we can model the pressure wave itself. An expanding sphere centered at ground zero is used to model the progression of the pressure wave through the city. To get a visual sense of the blast’s force, the color of the sphere will be a function of the pressure exerted by the pressure wave.</p>
|
||||
<p>The pressure exerted by the explosion in kilopascals (kPa) at various distances can be calculated using the U.S. Department of Defemse’s Blast Effects Computer, which allows users to input variables such as the TNT equivalent of the ordnance, storage method, and elevation. Though there are several estimates, the blast was likely equivalent to around 300 tons of TNT. The direct “incident pressure” of the pressure wave is shown in blue. However, pressure waves from explosions that occur on the ground are reflected upwards, amplifying the total pressure exerted by the blast. This “reflected pressure” is shown in orange:</p>
|
||||
<iframe title="Blast Overpressure and Distance " aria-label="Interactive line chart" id="datawrapper-chart-J1Pb1" src="https://datawrapper.dwcdn.net/J1Pb1/1/" scrolling="no" frameborder="0" style="width: 0; min-width: 100% !important; border: none;" height="400">
|
||||
</iframe>
|
||||
<script type="text/javascript">!function(){"use strict";window.addEventListener("message",(function(a){if(void 0!==a.data["datawrapper-height"])for(var e in a.data["datawrapper-height"]){var t=document.getElementById("datawrapper-chart-"+e)||document.querySelector("iframe[src*='"+e+"']");t&&(t.style.height=a.data["datawrapper-height"][e]+"px")}}))}();
|
||||
</script>
|
||||
<p>For reference, 137 kPa results in 99% fatalities, 68 kPa is enough to cause structural damage to most buildings, and 20 kPa results in serious injuries. 1-6 kPa is enough to break an average window. At 1km, the reflected pressure of the blast (18 kPa) was still enough to seriously injure. Precisely calculating the force exerted by an explosion is exceptionally complicated, however, so these numbers should be treated as rough estimates. Further analysis of the damage caused by blasts blast can be derived from the UN’s Explosion Consequences Analysis calculator which provides distance values for different types of damage and injuries.</p>
|
||||
</script>
|
||||
<p>For reference, 137 kPa results in 99% fatalities, 68 kPa is enough to cause structural damage to most buildings, and 20 kPa results in serious injuries. 1-6 kPa is enough to break an average window. At 1km, the reflected pressure of the blast (18 kPa) was still enough to seriously injure. Precisely calculating the force exerted by an explosion is exceptionally complicated, however, so these numbers should be treated as rough estimates. Further analysis of the damage caused by the blast can be derived from the UN’s Explosion Consequences Analysis calculator which provides distance values for different types of damage and injuries.</p>
|
||||
<p>Linking the values in this graph to the color of the pressure wave sphere provides a visual representation of the blast’s force as it expands. An RGB color scale corresponds to the blast’s overpressure at three threshold values.</p>
|
||||
<p><img src="./images/beirut/beirut.gif" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/beirut.gif" class="img-fluid"></p>
|
||||
<p>By keeping the lighting object from the viewshed analysis and placing it within the expanding sphere of the pressure wave, we combine two key pieces of information: the pressure exerted by the blast (the color of the sphere), and the level of directional exposure (brightness).</p>
|
||||
<p>Now, referring back to the two geolocated twitter videos from earlier, we can recreate the blast in our 3D model and get some new insights. Below is a side-by-side comparison of the first video and the 3D model:</p>
|
||||
<p><img src="./images/beirut/GIF_5.gif" class="img-fluid"></p>
|
||||
<p>Judging by the twitter video alone, it would be very hard to tell the fate of the person filming or the damage caused to the building that they were in. However, the 3D model shows that despite having an unobstructed view of the explosion, the incident pressure of the pressure wave had decreased significtantly by the time it reached the viewing point. The blue-green color corresponds to roughly 15 kPa– enough to injure and break windows, but not enough to cause structural damage to the building.</p>
|
||||
<p><img src="../images/beirut/GIF_5.gif" class="img-fluid"></p>
|
||||
<p>Judging by the twitter video alone, it would be very hard to tell the fate of the person filming or the damage caused to the building that they were in. However, the 3D model shows that despite having an unobstructed view of the explosion, the incident pressure of the pressure wave had decreased significantly by the time it reached the viewing point. The blue-green color corresponds to roughly 15 kPa– enough to injure and break windows, but not enough to cause structural damage to the building.</p>
|
||||
<p>The second twitter video was taken slightly closer to ground zero, but the view was partially obstructed by the grain silo:</p>
|
||||
<p><img src="./images/beirut/GIF_6.gif" class="img-fluid"></p>
|
||||
<p><img src="../images/beirut/GIF_6.gif" class="img-fluid"></p>
|
||||
<p>Though the pressure wave probably exerted more pressure compared to the first angle, the partial obstruction of the grain silo likely tempered the force of the blast.</p>
|
||||
</section>
|
||||
<section id="assessing-damage-to-the-skyline-tower" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="assessing-damage-to-the-skyline-tower">Assessing Damage to the Skyline Tower</h3>
|
||||
<p>As a concrete example of how this approach can be used to assess damage (or predict it, if one had the foresight), let us consider the Skyline Tower, pictured below following the explosion:</p>
|
||||
<p><img src="./images/beirut/IMG_8.png" class="img-fluid"></p>
|
||||
<p>This partial side view shows two faces of the building, labelled “A” and “B” above. Side A was nearly perpendicular to the blast, and just 600 m from ground zero. Based on the previous modeling, the pressure wave exerted roughly 40 kPa on this side of the building. The corner where sides A and B meet, highlighted in green, shows total destruction of windows, removal of most siding panels, and structural damage. The back corner, highlighted in red, shows many windows still intact, indicating that the maximum overpressure on this side of the building likely didn’t exeed 10 kPa. In other words, standing on the front balcony would likely have led to serious injury but standing on the back balcony would have been relatively safe.</p>
|
||||
<p><img src="../images/beirut/IMG_8.png" class="img-fluid"></p>
|
||||
<p>This partial side view shows two faces of the building, labeled “A” and “B” above. Side A was nearly perpendicular to the blast, and just 600m from ground zero. Based on the previous modeling, the pressure wave exerted roughly 40 kPa on this side of the building. The corner where sides A and B meet, highlighted in green, shows total destruction of windows, removal of most siding panels, and structural damage. The back corner, highlighted in red, shows many windows still intact, indicating that the maximum overpressure on this side of the building likely didn’t exceed 10 kPa. In other words, standing on the front balcony would likely have led to serious injury but standing on the back balcony would have been relatively safe.</p>
|
||||
<p>The animation below shows the Skyline Tower as it is hit by the pressure wave, with sides A and B labeled:</p>
|
||||
<p><img src="./images/beirut/GIF_7.gif" class="img-fluid"></p>
|
||||
<p>The bright green color of the pressure wave indicates a strong likelihood of structural damage. Side A can be seen taking a direct hit, while side B is angled slighly away. Despite not being directly exposed to the blast, it likely still took reflective damage from some of the neighbouring buildings. Both the incident overpressure indicated by the color of the sphere, as well as the relative brightness of sides A and B both correspond closely to the observed damage taken by the Skyline Tower.</p>
|
||||
<p><img src="../images/beirut/GIF_7.gif" class="img-fluid"></p>
|
||||
<p>The bright green color of the pressure wave indicates a strong likelihood of structural damage. Side A can be seen taking a direct hit, while side B is angled slightly away. Despite not being directly exposed to the blast, it likely still took reflective damage from some of the neighboring buildings. Both the incident overpressure indicated by the color of the sphere, as well as the relative brightness of sides A and B both correspond closely to the observed damage taken by the Skyline Tower.</p>
|
||||
</section>
|
||||
<section id="further-research" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="further-research">Further Research</h3>
|
||||
<p>Though satellite imagery analysis is an indispensable tool in disaster response, it has limitations. Urban blast damage in particular is difficult to assess accurately because it is highly directional and much of it cannot be seen from a bird’s eye view. Using free and open source tools, an interactive 3D model of an urban explosion can be generated, allowing for a highly detailed investigation of directional blast damage. This can be achieved in three steps:</p>
|
||||
<p>First, creating a 3D model of the urban area using Blender and Open Street Maps data. Second, conducting a viewshed analysis using lighting objects to gauge levels of unobstructed exposure to the pressure wave. Third, modeling the explosion using geolocated videos of the event and ordnance calculators. For added detail, a dynamic wave analysis can be used to more precisely model how the pressure wave interacts with buildings.</p>
|
||||
<p>Once properly modeled, the explosion can be viewed from any angle in the city. The viewshed analysis can be calibrated more finely by ground-truthing various damage levels (e.g. broken windows) at different locations. In the absence of an official address registry in Beirut, OSM is already being used by the Lebanese Red Cross (donate here) to conduct neighborhood surveys assessing blast damage. As such, this type of damage analysis can quickly be integrated into relief efforts, adapted to model disasters in different cities, and can even be used to simulate the destructive potential of hypothetical explosions to promote readiness.</p>
|
||||
<p>Speacial thanks to my nuclear physicist brother, Sean, for making sure I didn’t commit too many crimes against Physics.</p>
|
||||
|
||||
|
||||
</section>
|
||||
@@ -950,13 +970,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./ships.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text">Ship Detection</span>
|
||||
<a href="../chapters/C2_Refineries.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Refinery Identification</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./object_detection.html" class="pagination-link">
|
||||
<span class="nav-page-text">Object Detection</span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/C4_Ships.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Ship Detection</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - Ship Detection</title>
|
||||
<title>Remote Sensing for OSINT - 10 Ship Detection</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -86,27 +86,27 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./blast.html" rel="next">
|
||||
<link href="./refineries.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/C5_Object_Detection.html" rel="next">
|
||||
<link href="../chapters/C3_Blast.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -146,7 +146,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title">Ship Detection</h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Ship Detection</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -158,24 +158,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -218,17 +218,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -243,22 +243,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -273,27 +273,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link active">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -327,14 +327,14 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</ul></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/ships.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/C4_Ships.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content page-columns page-full" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block">Ship Detection</h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Ship Detection</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -349,7 +349,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
|
||||
</header>
|
||||
|
||||
<p>There’s a huge amount of data available on the internet about ship movements, most of which draw on the Automatic Identification System (AIS) which is a system that uses radio to broadcast the identity, position, course, speed, and other data about ships. <a href="https://www.marinetraffic.com/en/ais-api-services">MarineTraffic</a>, for example, provides an API that allows you to query the location of ships in real time as well as historical vessel tracks and lots of other useful data. Unfortunately most sources of AIS data are paywalled, and AIS can be turned off or manipulated to hide the identity or position of the ship. In fact, most of the stuff we’re interested in investigating probably happens when AIS is turned off.</p>
|
||||
<p>There’s a huge amount of data available on the internet about ship movements, most of which draws on the Automatic Identification System (AIS) which uses radio to broadcast the identity, position, course, speed and other data about ships. <a href="https://www.marinetraffic.com/en/ais-api-services">MarineTraffic</a>, for example, provides an API that allows you to query the location of ships in real time as well as historical vessel tracks and lots of other useful data. Unfortunately most sources of AIS data are paywalled, and AIS can be turned off or manipulated to hide the identity or position of the ship. In fact, most of the stuff we’re interested in investigating probably happens when AIS is turned off.</p>
|
||||
<p>Though ships can hide by turning off their AIS transponders, they can’t hide from satellites. In this tutorial, we’re going to build an application that uses Synthetic Aperture Radar (SAR) from the European Space Agency’s Sentinel-1 satellite to automatically identify ships, regardless of whether they’ve got their transponders turned on or off. Here’s the finished application:</p>
|
||||
<div class="column-page">
|
||||
<iframe src="https://ollielballinger.users.earthengine.app/view/shipdetection" width="100%" height="700px">
|
||||
@@ -363,9 +363,9 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<li>A map on the right that displays the results</li>
|
||||
</ol>
|
||||
<p>The control panel has a date slider that allows the user to load imagery from a particular year. Below that is a graph that shows the number of ships detected over time within that year. A slider underneath the graph lets us toggle the sensitivity of the ship detection process. Finally, a button at the bottom lets the user draw their own area of interest on the map, and the app will automatically detect ships within that area.</p>
|
||||
<p>The map panel visualizes the results of the ship detection process and has three layers. The bottom layer is the Sentinel-1 image that we’re using to detect ships; it’s blue/purple, and if you zoom in and look closley you can see bright specks in the sea, which are ships. When Sentinel-1 sends a pulse of radio waves onto a flat surface like the sea, there is very little to reflect the waves back to the satellite– they just bounce off into space. A low return signal means we’ll see a darker color on our map. But when the raio waves hit a ship they are reflected back to the satelite and generate a higher return signal, and therefore a much brighter color. The second layer on the map displays a bunch of green points; each one of these is a detected ship. The last layer shows the red outline of the area of interest that the user drew on the map. You can zoom in on the map by holding down the command button and scrolling up and down.</p>
|
||||
<p>The map panel visualizes the results of the ship detection process and has three layers. The bottom layer is the Sentinel-1 image that we’re using to detect ships; it’s blue/purple, and if you zoom in and look cloesly you can see bright specks in the sea, which are ships. When Sentinel-1 sends a pulse of radio waves onto a flat surface like the sea, there is very little to reflect the waves back to the satellite – they just bounce off into space. A low return signal means we’ll see a darker color on our map. But when the radio waves hit a ship they are reflected back to the satellite and generate a higher return signal, and therefore a much brighter color. The second layer on the map displays a bunch of green points; each one of these is a detected ship. The last layer shows the red outline of the area of interest that the user drew on the map. You can zoom in on the map by holding down the command button and scrolling up and down.</p>
|
||||
<p>When the application is first loaded it is centered on an area just north of the Suez Canal, and is analyzing imagery from 2021. We can see a bunch of green dots in the AOI, which is the main waiting area for ships waiting to transit the canal. It’s a bit crowded because it’s visualizing all of the ships detected in the entire year. We can display imagery from a single day by clicking on a point in the graph on the left, which you will notice displays a huge spike in the number of ships detected around March.</p>
|
||||
<p>You might remember that on March 23rd, 2021, the Ever Given– a 400m long container ship– got stuck in the Suez Canal. The ship was blocking the canal for six days, and it’s estimated that it cost the global economy $400 million per day. If you click on the tip of the spike on March 30th, you can see backup of around 150 ships waiting for the canal to be cleared. You can also zoom in on a particular date range by scrolling and dragging on the graph. If you zoom in on the spike, you can then select imagery from early April to compare the number of ships in the waiting area after the blockage was cleared. In normal times we can see a regular pattern in the number of ships in the waiting area ranging between 15 and 40 ships.</p>
|
||||
<p>You might remember that on March 23rd, 2021, the Ever Given – a 400m long container ship – got stuck in the Suez Canal. The ship was blocking the canal for six days, and it’s estimated that it cost the global economy $400 million per day. If you click on the tip of the spike on March 30th, you can see a backup of around 150 ships waiting for the canal to be cleared. You can also zoom in on a particular date range by scrolling and dragging on the graph. If you zoom in on the spike, you can then select imagery from early April to compare the number of ships in the waiting area after the blockage was cleared. In normal times we can see a regular pattern in the number of ships in the waiting area ranging between 15 and 40 ships.</p>
|
||||
<p>If you’re closely zoomed in to the map and load imagery from different days by clicking on the graph, you can compare the bright spots on the Sentinel image and the green dots. The ship detection process is pretty accurate, and we typically see one green dot per ship. However, you may notice that we occasionally miss a ship. This is because the ship detection process is based on a threshold, and if the ship is too small it may not generate a high enough return signal to be detected. You can increase the sensitivity of the ship detection process by moving the slider below the graph. This will increase the number of ships detected, but it may also increase the number of false positives.</p>
|
||||
<p>The next section focuses on building this application. After that, we’ll have a look at a few different use cases for this sort of maritime surveillance.</p>
|
||||
</section>
|
||||
@@ -379,25 +379,28 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setOptions</span>(<span class="st">"Hybrid"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="bu">Map</span><span class="op">.</span><span class="fu">setControlVisibility</span>({ <span class="dt">all</span><span class="op">:</span> <span class="kw">false</span> })<span class="op">;</span></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="co">// Import the Digital Surface Model (DSM) from the ALOS World 3D-30 dataset</span></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> dem <span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">"JAXA/ALOS/AW3D30/V3_2"</span>)<span class="op">.</span><span class="fu">mean</span>()<span class="op">.</span><span class="fu">select</span>(<span class="st">"DSM"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="co">// Import the Sentinel 1 dataset</span></span>
|
||||
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> s1 <span class="op">=</span> ee</span>
|
||||
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">"COPERNICUS/S1_GRD"</span>)</span>
|
||||
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">listContains</span>(<span class="st">"transmitterReceiverPolarisation"</span><span class="op">,</span> <span class="st">"VV"</span>))</span>
|
||||
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">eq</span>(<span class="st">"instrumentMode"</span><span class="op">,</span> <span class="st">"IW"</span>))</span>
|
||||
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">sort</span>(<span class="st">"system:time_start"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a><span class="co">// Define the default area of interest</span></span>
|
||||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> suez <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>([</span>
|
||||
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a> [</span>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.17388584692775</span><span class="op">,</span> <span class="fl">31.59541178442045</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.17388584692775</span><span class="op">,</span> <span class="fl">31.327159861902278</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.4787564523965</span><span class="op">,</span> <span class="fl">31.327159861902278</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.4787564523965</span><span class="op">,</span> <span class="fl">31.59541178442045</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a> ]<span class="op">,</span></span>
|
||||
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a>])<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="co">// Import the Digital Surface Model (DSM) from the ALOS World 3D-30 dataset</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> dem <span class="op">=</span> ee<span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">"JAXA/ALOS/AW3D30/V3_2"</span>)<span class="op">.</span><span class="fu">mean</span>()<span class="op">.</span><span class="fu">select</span>(<span class="st">"DSM"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-10"><a href="#cb1-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-11"><a href="#cb1-11" aria-hidden="true" tabindex="-1"></a><span class="co">// Import the Sentinel 1 dataset</span></span>
|
||||
<span id="cb1-12"><a href="#cb1-12" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> s1 <span class="op">=</span> ee</span>
|
||||
<span id="cb1-13"><a href="#cb1-13" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">ImageCollection</span>(<span class="st">"COPERNICUS/S1_GRD"</span>)</span>
|
||||
<span id="cb1-14"><a href="#cb1-14" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">listContains</span>(<span class="st">"transmitterReceiverPolarisation"</span><span class="op">,</span> <span class="st">"VV"</span>))</span>
|
||||
<span id="cb1-15"><a href="#cb1-15" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">filter</span>(ee<span class="op">.</span><span class="at">Filter</span><span class="op">.</span><span class="fu">eq</span>(<span class="st">"instrumentMode"</span><span class="op">,</span> <span class="st">"IW"</span>))</span>
|
||||
<span id="cb1-16"><a href="#cb1-16" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">sort</span>(<span class="st">"system:time_start"</span>)<span class="op">;</span></span>
|
||||
<span id="cb1-17"><a href="#cb1-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-18"><a href="#cb1-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-19"><a href="#cb1-19" aria-hidden="true" tabindex="-1"></a><span class="co">// Define the default area of interest</span></span>
|
||||
<span id="cb1-20"><a href="#cb1-20" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> suez <span class="op">=</span> ee<span class="op">.</span><span class="at">Geometry</span><span class="op">.</span><span class="fu">Polygon</span>([</span>
|
||||
<span id="cb1-21"><a href="#cb1-21" aria-hidden="true" tabindex="-1"></a> [</span>
|
||||
<span id="cb1-22"><a href="#cb1-22" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.17388584692775</span><span class="op">,</span> <span class="fl">31.59541178442045</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-23"><a href="#cb1-23" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.17388584692775</span><span class="op">,</span> <span class="fl">31.327159861902278</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-24"><a href="#cb1-24" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.4787564523965</span><span class="op">,</span> <span class="fl">31.327159861902278</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-25"><a href="#cb1-25" aria-hidden="true" tabindex="-1"></a> [<span class="fl">32.4787564523965</span><span class="op">,</span> <span class="fl">31.59541178442045</span>]<span class="op">,</span></span>
|
||||
<span id="cb1-26"><a href="#cb1-26" aria-hidden="true" tabindex="-1"></a> ]<span class="op">,</span></span>
|
||||
<span id="cb1-27"><a href="#cb1-27" aria-hidden="true" tabindex="-1"></a>])<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now that we’ve gotten that out of the way, we can move on to the actual detection of ships.</p>
|
||||
</section>
|
||||
<section id="ship-detection" class="level2">
|
||||
@@ -408,69 +411,77 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the area of interest from the drawing tools widget. </span></span>
|
||||
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> aoi <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">getEeObject</span>()<span class="op">;</span></span>
|
||||
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a> <span class="co">// Clip the image to the area of interest</span></span>
|
||||
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a> <span class="co">// Select the VV polarization </span></span>
|
||||
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter areas where the VV value is greater than 0</span></span>
|
||||
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> cutoff <span class="op">=</span> img<span class="op">.</span><span class="fu">clip</span>(aoi)<span class="op">.</span><span class="fu">select</span>(<span class="st">"VV"</span>)<span class="op">.</span><span class="fu">gt</span>(<span class="dv">0</span>)</span>
|
||||
<span id="cb2-9"><a href="#cb2-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-10"><a href="#cb2-10" aria-hidden="true" tabindex="-1"></a> <span class="co">// Convert the raster image to a FeatureCollection of points</span></span>
|
||||
<span id="cb2-11"><a href="#cb2-11" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> points <span class="op">=</span> cutoff<span class="op">.</span><span class="fu">reduceToVectors</span>({</span>
|
||||
<span id="cb2-12"><a href="#cb2-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">geometry</span><span class="op">:</span> aoi<span class="op">,</span></span>
|
||||
<span id="cb2-13"><a href="#cb2-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> scaleSlider<span class="op">.</span><span class="fu">getValue</span>()<span class="op">,</span></span>
|
||||
<span id="cb2-14"><a href="#cb2-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">geometryType</span><span class="op">:</span> <span class="st">"centroid"</span><span class="op">,</span></span>
|
||||
<span id="cb2-15"><a href="#cb2-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">eightConnected</span><span class="op">:</span> <span class="kw">true</span><span class="op">,</span></span>
|
||||
<span id="cb2-16"><a href="#cb2-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">maxPixels</span><span class="op">:</span> <span class="dv">1653602926</span><span class="op">,</span></span>
|
||||
<span id="cb2-17"><a href="#cb2-17" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb2-18"><a href="#cb2-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-19"><a href="#cb2-19" aria-hidden="true" tabindex="-1"></a> <span class="co">// Set the number of ships detected in the image as a property called "count"</span></span>
|
||||
<span id="cb2-20"><a href="#cb2-20" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> count <span class="op">=</span> points<span class="op">.</span><span class="fu">size</span>()<span class="op">;</span></span>
|
||||
<span id="cb2-21"><a href="#cb2-21" aria-hidden="true" tabindex="-1"></a> <span class="co">// Set the date of the image as a property called "system:time_start"</span></span>
|
||||
<span id="cb2-22"><a href="#cb2-22" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> date <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(img<span class="op">.</span><span class="fu">get</span>(<span class="st">"system:time_start"</span>))<span class="op">;</span></span>
|
||||
<span id="cb2-23"><a href="#cb2-23" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> points<span class="op">.</span><span class="fu">set</span>(<span class="st">"count"</span><span class="op">,</span> count)<span class="op">.</span><span class="fu">set</span>(<span class="st">"system:time_start"</span><span class="op">,</span> date)<span class="op">;</span></span>
|
||||
<span id="cb2-24"><a href="#cb2-24" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a> <span class="co">// Clip the image to the area of interest</span></span>
|
||||
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a> <span class="co">// Select the VV polarization </span></span>
|
||||
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter areas where the VV value is greater than 0</span></span>
|
||||
<span id="cb2-9"><a href="#cb2-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> cutoff <span class="op">=</span> img<span class="op">.</span><span class="fu">clip</span>(aoi)<span class="op">.</span><span class="fu">select</span>(<span class="st">"VV"</span>)<span class="op">.</span><span class="fu">gt</span>(<span class="dv">0</span>)</span>
|
||||
<span id="cb2-10"><a href="#cb2-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-11"><a href="#cb2-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-12"><a href="#cb2-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Convert the raster image to a FeatureCollection of points</span></span>
|
||||
<span id="cb2-13"><a href="#cb2-13" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> points <span class="op">=</span> cutoff<span class="op">.</span><span class="fu">reduceToVectors</span>({</span>
|
||||
<span id="cb2-14"><a href="#cb2-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">geometry</span><span class="op">:</span> aoi<span class="op">,</span></span>
|
||||
<span id="cb2-15"><a href="#cb2-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">scale</span><span class="op">:</span> scaleSlider<span class="op">.</span><span class="fu">getValue</span>()<span class="op">,</span></span>
|
||||
<span id="cb2-16"><a href="#cb2-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">geometryType</span><span class="op">:</span> <span class="st">"centroid"</span><span class="op">,</span></span>
|
||||
<span id="cb2-17"><a href="#cb2-17" aria-hidden="true" tabindex="-1"></a> <span class="dt">eightConnected</span><span class="op">:</span> <span class="kw">true</span><span class="op">,</span></span>
|
||||
<span id="cb2-18"><a href="#cb2-18" aria-hidden="true" tabindex="-1"></a> <span class="dt">maxPixels</span><span class="op">:</span> <span class="dv">1653602926</span><span class="op">,</span></span>
|
||||
<span id="cb2-19"><a href="#cb2-19" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb2-20"><a href="#cb2-20" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-21"><a href="#cb2-21" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb2-22"><a href="#cb2-22" aria-hidden="true" tabindex="-1"></a> <span class="co">// Set the number of ships detected in the image as a property called "count"</span></span>
|
||||
<span id="cb2-23"><a href="#cb2-23" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> count <span class="op">=</span> points<span class="op">.</span><span class="fu">size</span>()<span class="op">;</span></span>
|
||||
<span id="cb2-24"><a href="#cb2-24" aria-hidden="true" tabindex="-1"></a> <span class="co">// Set the date of the image as a property called "system:time_start"</span></span>
|
||||
<span id="cb2-25"><a href="#cb2-25" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> date <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(img<span class="op">.</span><span class="fu">get</span>(<span class="st">"system:time_start"</span>))<span class="op">;</span></span>
|
||||
<span id="cb2-26"><a href="#cb2-26" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> points<span class="op">.</span><span class="fu">set</span>(<span class="st">"count"</span><span class="op">,</span> count)<span class="op">.</span><span class="fu">set</span>(<span class="st">"system:time_start"</span><span class="op">,</span> date)<span class="op">;</span></span>
|
||||
<span id="cb2-27"><a href="#cb2-27" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The <code>count</code> and <code>system:time_start</code> properties are used to create the graph of daily ship counts and allow the resulting vector (point) data to interact with the date slider widget. An important detail here is that the “scale” parameter of the <code>reduceToVectors</code> function is set to the value of the scale slider widget. This allows the user to adjust the resolution of the ship detection process; a smaller value will allow us to detect smaller ships.</p>
|
||||
</section>
|
||||
<section id="visualization" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="visualization">Visualization</h2>
|
||||
<p>The <code>viz</code> function is responsible for displaying the results of the ship detection process. It takes the area of interest, the vector data, and the Sentinel 1 image as inputs. Nothing super complicated here; we’re just creating three layers and adding them to the map in order: the underlying Sentinel-1 image raster, the ship vector data in green, and the area of interest outline in red. We’re using the <code>Map.layers().set()</code> function to replace the existing layers with the new ones, rather than addine new ones each time.</p>
|
||||
<p>The <code>viz</code> function is responsible for displaying the results of the ship detection process. It takes the area of interest, the vector data, and the Sentinel 1 image as inputs. Nothing super complicated here; we’re just creating three layers and adding them to the map in order: the underlying Sentinel-1 image raster, the ship vector data in green, and the area of interest outline in red. We’re using the <code>Map.layers().set()</code> function to replace the existing layers with the new ones, rather than adding new ones each time.</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">viz</span>(aoi<span class="op">,</span> vectors<span class="op">,</span> s1Filtered) {</span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create an empty image into which to paint the features, cast to byte.</span></span>
|
||||
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> empty <span class="op">=</span> ee<span class="op">.</span><span class="fu">Image</span>()<span class="op">.</span><span class="fu">byte</span>()<span class="op">;</span></span>
|
||||
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a> <span class="co">// Paint all the polygon edges with the same number and width, display.</span></span>
|
||||
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> outline <span class="op">=</span> empty<span class="op">.</span><span class="fu">paint</span>({</span>
|
||||
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">featureCollection</span><span class="op">:</span> aoi<span class="op">,</span></span>
|
||||
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">color</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">width</span><span class="op">:</span> <span class="dv">3</span><span class="op">,</span></span>
|
||||
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a layer for the area of interest in red</span></span>
|
||||
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> aoi_layer <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">Layer</span>(outline<span class="op">,</span> { <span class="dt">palette</span><span class="op">:</span> <span class="st">"red"</span> }<span class="op">,</span> <span class="st">"AOI"</span>)<span class="op">;</span></span>
|
||||
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-15"><a href="#cb3-15" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a layer for the vector data in green</span></span>
|
||||
<span id="cb3-16"><a href="#cb3-16" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> vectorLayer <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">Layer</span>(</span>
|
||||
<span id="cb3-17"><a href="#cb3-17" aria-hidden="true" tabindex="-1"></a> vectors<span class="op">.</span><span class="fu">flatten</span>()<span class="op">,</span></span>
|
||||
<span id="cb3-18"><a href="#cb3-18" aria-hidden="true" tabindex="-1"></a> { <span class="dt">color</span><span class="op">:</span> <span class="st">"#39ff14"</span> }<span class="op">,</span></span>
|
||||
<span id="cb3-19"><a href="#cb3-19" aria-hidden="true" tabindex="-1"></a> <span class="st">"Vectors"</span></span>
|
||||
<span id="cb3-20"><a href="#cb3-20" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb3-21"><a href="#cb3-21" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-22"><a href="#cb3-22" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a layer for the Sentinel 1 image in false color</span></span>
|
||||
<span id="cb3-23"><a href="#cb3-23" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> sarLayer <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">Layer</span>(</span>
|
||||
<span id="cb3-24"><a href="#cb3-24" aria-hidden="true" tabindex="-1"></a> s1Filtered<span class="op">,</span></span>
|
||||
<span id="cb3-25"><a href="#cb3-25" aria-hidden="true" tabindex="-1"></a> { <span class="dt">min</span><span class="op">:</span> [<span class="op">-</span><span class="dv">25</span><span class="op">,</span> <span class="op">-</span><span class="dv">20</span><span class="op">,</span> <span class="op">-</span><span class="dv">25</span>]<span class="op">,</span> <span class="dt">max</span><span class="op">:</span> [<span class="dv">0</span><span class="op">,</span> <span class="dv">10</span><span class="op">,</span> <span class="dv">0</span>]<span class="op">,</span> <span class="dt">opacity</span><span class="op">:</span> <span class="fl">0.8</span> }<span class="op">,</span></span>
|
||||
<span id="cb3-26"><a href="#cb3-26" aria-hidden="true" tabindex="-1"></a> <span class="st">"SAR"</span></span>
|
||||
<span id="cb3-27"><a href="#cb3-27" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb3-28"><a href="#cb3-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-29"><a href="#cb3-29" aria-hidden="true" tabindex="-1"></a> <span class="co">// Add the layers in order</span></span>
|
||||
<span id="cb3-30"><a href="#cb3-30" aria-hidden="true" tabindex="-1"></a> <span class="bu">Map</span><span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">0</span><span class="op">,</span> sarLayer)<span class="op">;</span></span>
|
||||
<span id="cb3-31"><a href="#cb3-31" aria-hidden="true" tabindex="-1"></a> <span class="bu">Map</span><span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">1</span><span class="op">,</span> vectorLayer)<span class="op">;</span></span>
|
||||
<span id="cb3-32"><a href="#cb3-32" aria-hidden="true" tabindex="-1"></a> <span class="bu">Map</span><span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">2</span><span class="op">,</span> aoi_layer)<span class="op">;</span></span>
|
||||
<span id="cb3-33"><a href="#cb3-33" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>We want a function to handle the visualization because there are two different situations in which we’re going to visualize results, and we dont want to repeat our code. The first situation is when the user draws a new area of interest, moves the date slider, or alters the scale. In this case, we want to visualize the results of the ship detection process for the entire year’s worth of Sentinel-1 imagery. The second situation is when the user clicks on the chart to analyze a particular day. In this case, we obviously only want to visualize the results of the ship detection process on that day. With this function, we can simply pass the appropriately filtered versions of the Sentinel-1 image and vector data to the function, and it will visualize the results, rather than having to write the same code twice.</p>
|
||||
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a> <span class="co">// Paint all the polygon edges with the same number and width, display.</span></span>
|
||||
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> outline <span class="op">=</span> empty<span class="op">.</span><span class="fu">paint</span>({</span>
|
||||
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">featureCollection</span><span class="op">:</span> aoi<span class="op">,</span></span>
|
||||
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">color</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">width</span><span class="op">:</span> <span class="dv">3</span><span class="op">,</span></span>
|
||||
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a layer for the area of interest in red</span></span>
|
||||
<span id="cb3-15"><a href="#cb3-15" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> aoi_layer <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">Layer</span>(outline<span class="op">,</span> { <span class="dt">palette</span><span class="op">:</span> <span class="st">"red"</span> }<span class="op">,</span> <span class="st">"AOI"</span>)<span class="op">;</span></span>
|
||||
<span id="cb3-16"><a href="#cb3-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-17"><a href="#cb3-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-18"><a href="#cb3-18" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a layer for the vector data in green</span></span>
|
||||
<span id="cb3-19"><a href="#cb3-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> vectorLayer <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">Layer</span>(</span>
|
||||
<span id="cb3-20"><a href="#cb3-20" aria-hidden="true" tabindex="-1"></a> vectors<span class="op">.</span><span class="fu">flatten</span>()<span class="op">,</span></span>
|
||||
<span id="cb3-21"><a href="#cb3-21" aria-hidden="true" tabindex="-1"></a> { <span class="dt">color</span><span class="op">:</span> <span class="st">"#39ff14"</span> }<span class="op">,</span></span>
|
||||
<span id="cb3-22"><a href="#cb3-22" aria-hidden="true" tabindex="-1"></a> <span class="st">"Vectors"</span></span>
|
||||
<span id="cb3-23"><a href="#cb3-23" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb3-24"><a href="#cb3-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-25"><a href="#cb3-25" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-26"><a href="#cb3-26" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a layer for the Sentinel 1 image in false color</span></span>
|
||||
<span id="cb3-27"><a href="#cb3-27" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> sarLayer <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">Layer</span>(</span>
|
||||
<span id="cb3-28"><a href="#cb3-28" aria-hidden="true" tabindex="-1"></a> s1Filtered<span class="op">,</span></span>
|
||||
<span id="cb3-29"><a href="#cb3-29" aria-hidden="true" tabindex="-1"></a> { <span class="dt">min</span><span class="op">:</span> [<span class="op">-</span><span class="dv">25</span><span class="op">,</span> <span class="op">-</span><span class="dv">20</span><span class="op">,</span> <span class="op">-</span><span class="dv">25</span>]<span class="op">,</span> <span class="dt">max</span><span class="op">:</span> [<span class="dv">0</span><span class="op">,</span> <span class="dv">10</span><span class="op">,</span> <span class="dv">0</span>]<span class="op">,</span> <span class="dt">opacity</span><span class="op">:</span> <span class="fl">0.8</span> }<span class="op">,</span></span>
|
||||
<span id="cb3-30"><a href="#cb3-30" aria-hidden="true" tabindex="-1"></a> <span class="st">"SAR"</span></span>
|
||||
<span id="cb3-31"><a href="#cb3-31" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb3-32"><a href="#cb3-32" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-33"><a href="#cb3-33" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-34"><a href="#cb3-34" aria-hidden="true" tabindex="-1"></a> <span class="co">// Add the layers in order</span></span>
|
||||
<span id="cb3-35"><a href="#cb3-35" aria-hidden="true" tabindex="-1"></a> <span class="bu">Map</span><span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">0</span><span class="op">,</span> sarLayer)<span class="op">;</span></span>
|
||||
<span id="cb3-36"><a href="#cb3-36" aria-hidden="true" tabindex="-1"></a> <span class="bu">Map</span><span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">1</span><span class="op">,</span> vectorLayer)<span class="op">;</span></span>
|
||||
<span id="cb3-37"><a href="#cb3-37" aria-hidden="true" tabindex="-1"></a> <span class="bu">Map</span><span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">2</span><span class="op">,</span> aoi_layer)<span class="op">;</span></span>
|
||||
<span id="cb3-38"><a href="#cb3-38" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>We want a function to handle the visualization because there are two different situations in which we’re going to visualize results, and we don’t want to repeat our code. The first situation is when the user draws a new area of interest, moves the date slider or alters the scale. In this case, we want to visualize the results of the ship detection process for the entire year’s worth of Sentinel-1 imagery. The second situation is when the user clicks on the chart to analyze a particular day. In this case, we obviously only want to visualize the results of the ship detection process on that day. With this function, we can simply pass the appropriately filtered versions of the Sentinel-1 image and vector data to the function, and it will visualize the results, rather than having to write the same code twice.</p>
|
||||
</section>
|
||||
<section id="putting-it-all-together" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="putting-it-all-together">Putting it all together</h2>
|
||||
<p>Having defined a few helper functions to handle the visualization and ship detection process, we can now move on to the main function that will perform the analysis. This will be performed by the <code>daterangeVectors</code> function. In a nutshell, it read the user specified date range from the date slider widget, and filter the Sentinel 1 dataset to only include images within that period. Then, it will loop through each Sentinel-1 image from that year and apply the <code>getVectors</code> function to count the number of ships that fall within the area of interest and generate a dataset of points corresponding to detected ships. We’ll then use the <code>viz</code> function we just defined to visualize the all of the ship detections and Sentinel-1 images in the AOI during that year stacked on top of each other. Finally, we’ll create a chart based on the number of ships detected per day, and allow the user to click on the chart to visualize the results for a particular day.</p>
|
||||
<p>Having defined a few helper functions to handle the visualization and ship detection process, we can now move on to the main function that will perform the analysis. This will be performed by the <code>daterangeVectors</code> function. In a nutshell, it reads the user specified date range from the date slider widget, and filters the Sentinel 1 dataset to only include images within that period. Then, it will loop through each Sentinel-1 image from that year and apply the <code>getVectors</code> function to count the number of ships that fall within the area of interest and generate a dataset of points corresponding to detected ships. We’ll then use the <code>viz</code> function we just defined to visualize all of the ship detections and Sentinel-1 images in the AOI during that year stacked on top of each other. Finally, we’ll create a chart based on the number of ships detected per day, and allow the user to click on the chart to visualize the results for a particular day.</p>
|
||||
<div class="sourceCode" id="cb4"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> daterangeVectors <span class="op">=</span> <span class="kw">function</span> () {</span>
|
||||
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb4-3"><a href="#cb4-3" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the date range from the date slider widget.</span></span>
|
||||
@@ -479,57 +490,67 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb4-6"><a href="#cb4-6" aria-hidden="true" tabindex="-1"></a> ee<span class="op">.</span><span class="fu">Date</span>(dateSlider<span class="op">.</span><span class="fu">getValue</span>()[<span class="dv">1</span>])</span>
|
||||
<span id="cb4-7"><a href="#cb4-7" aria-hidden="true" tabindex="-1"></a> )<span class="op">;</span></span>
|
||||
<span id="cb4-8"><a href="#cb4-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the area of interest from the drawing tools widget.</span></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> aoi <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">getEeObject</span>()<span class="op">;</span></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Hide the user-drawn shape.</span></span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">setShown</span>(<span class="kw">false</span>)<span class="op">;</span></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the Sentinel 1 dataset to only include images within the date range, and within the area of interest.</span></span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> s1Filtered <span class="op">=</span> s1<span class="op">.</span><span class="fu">filterDate</span>(range<span class="op">.</span><span class="fu">start</span>()<span class="op">,</span> range<span class="op">.</span><span class="fu">end</span>())<span class="op">.</span><span class="fu">filterBounds</span>(aoi)<span class="op">;</span></span>
|
||||
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a> <span class="co">// Count the number of ships in each image using the getVectors function</span></span>
|
||||
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> vectors <span class="op">=</span> s1Filtered<span class="op">.</span><span class="fu">map</span>(getVectors)<span class="op">;</span></span>
|
||||
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a> <span class="co">// Use the viz function to visualize the results </span></span>
|
||||
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a> <span class="fu">viz</span>(aoi<span class="op">,</span> vectors<span class="op">,</span> s1Filtered<span class="op">.</span><span class="fu">max</span>()<span class="op">.</span><span class="fu">updateMask</span>(dem<span class="op">.</span><span class="fu">lte</span>(<span class="dv">0</span>)))<span class="op">;</span></span>
|
||||
<span id="cb4-9"><a href="#cb4-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-10"><a href="#cb4-10" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the area of interest from the drawing tools widget.</span></span>
|
||||
<span id="cb4-11"><a href="#cb4-11" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> aoi <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">getEeObject</span>()<span class="op">;</span></span>
|
||||
<span id="cb4-12"><a href="#cb4-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-13"><a href="#cb4-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-14"><a href="#cb4-14" aria-hidden="true" tabindex="-1"></a> <span class="co">// Hide the user-drawn shape.</span></span>
|
||||
<span id="cb4-15"><a href="#cb4-15" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">setShown</span>(<span class="kw">false</span>)<span class="op">;</span></span>
|
||||
<span id="cb4-16"><a href="#cb4-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-17"><a href="#cb4-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-18"><a href="#cb4-18" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the Sentinel 1 dataset to only include images within the date range, and within the area of interest.</span></span>
|
||||
<span id="cb4-19"><a href="#cb4-19" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> s1Filtered <span class="op">=</span> s1<span class="op">.</span><span class="fu">filterDate</span>(range<span class="op">.</span><span class="fu">start</span>()<span class="op">,</span> range<span class="op">.</span><span class="fu">end</span>())<span class="op">.</span><span class="fu">filterBounds</span>(aoi)<span class="op">;</span></span>
|
||||
<span id="cb4-20"><a href="#cb4-20" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb4-21"><a href="#cb4-21" aria-hidden="true" tabindex="-1"></a> <span class="co">// Count the number of ships in each image using the getVectors function</span></span>
|
||||
<span id="cb4-22"><a href="#cb4-22" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> vectors <span class="op">=</span> s1Filtered<span class="op">.</span><span class="fu">map</span>(getVectors)<span class="op">;</span></span>
|
||||
<span id="cb4-23"><a href="#cb4-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-24"><a href="#cb4-24" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a chart of the number of ships per day</span></span>
|
||||
<span id="cb4-25"><a href="#cb4-25" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> chart <span class="op">=</span> ui<span class="op">.</span><span class="at">Chart</span><span class="op">.</span><span class="at">feature</span></span>
|
||||
<span id="cb4-26"><a href="#cb4-26" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">byFeature</span>({</span>
|
||||
<span id="cb4-27"><a href="#cb4-27" aria-hidden="true" tabindex="-1"></a> <span class="dt">features</span><span class="op">:</span> vectors<span class="op">,</span></span>
|
||||
<span id="cb4-28"><a href="#cb4-28" aria-hidden="true" tabindex="-1"></a> <span class="dt">xProperty</span><span class="op">:</span> <span class="st">"system:time_start"</span><span class="op">,</span></span>
|
||||
<span id="cb4-29"><a href="#cb4-29" aria-hidden="true" tabindex="-1"></a> <span class="dt">yProperties</span><span class="op">:</span> [<span class="st">"count"</span>]<span class="op">,</span></span>
|
||||
<span id="cb4-30"><a href="#cb4-30" aria-hidden="true" tabindex="-1"></a> })</span>
|
||||
<span id="cb4-31"><a href="#cb4-31" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">setOptions</span>({</span>
|
||||
<span id="cb4-32"><a href="#cb4-32" aria-hidden="true" tabindex="-1"></a> <span class="dt">title</span><span class="op">:</span> <span class="st">"Daily Number of Ships in Area of Interest"</span><span class="op">,</span></span>
|
||||
<span id="cb4-33"><a href="#cb4-33" aria-hidden="true" tabindex="-1"></a> <span class="dt">vAxis</span><span class="op">:</span> { <span class="dt">title</span><span class="op">:</span> <span class="st">"Ship Count"</span> }<span class="op">,</span></span>
|
||||
<span id="cb4-34"><a href="#cb4-34" aria-hidden="true" tabindex="-1"></a> <span class="dt">explorer</span><span class="op">:</span> { <span class="dt">axis</span><span class="op">:</span> <span class="st">"horizontal"</span> }<span class="op">,</span></span>
|
||||
<span id="cb4-35"><a href="#cb4-35" aria-hidden="true" tabindex="-1"></a> <span class="dt">lineWidth</span><span class="op">:</span> <span class="dv">2</span><span class="op">,</span></span>
|
||||
<span id="cb4-36"><a href="#cb4-36" aria-hidden="true" tabindex="-1"></a> <span class="dt">series</span><span class="op">:</span> <span class="st">"Area of Interest"</span><span class="op">,</span></span>
|
||||
<span id="cb4-37"><a href="#cb4-37" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-38"><a href="#cb4-38" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-39"><a href="#cb4-39" aria-hidden="true" tabindex="-1"></a> <span class="co">// Add the chart at a fixed position, so that new charts overwrite older ones.</span></span>
|
||||
<span id="cb4-40"><a href="#cb4-40" aria-hidden="true" tabindex="-1"></a> controlPanel<span class="op">.</span><span class="fu">widgets</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">4</span><span class="op">,</span> chart)<span class="op">;</span></span>
|
||||
<span id="cb4-41"><a href="#cb4-41" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-42"><a href="#cb4-42" aria-hidden="true" tabindex="-1"></a> <span class="co">// Add a click handler to the chart to filter the map by day.</span></span>
|
||||
<span id="cb4-43"><a href="#cb4-43" aria-hidden="true" tabindex="-1"></a> chart<span class="op">.</span><span class="fu">onClick</span>(filterDay)<span class="op">;</span></span>
|
||||
<span id="cb4-44"><a href="#cb4-44" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>There’s one function referenced above– <code>filterDay</code>– that we haven’t defined yet. This function is called when the user clicks on the chart to analyze a particular day. It takes the date of the clicked day as an input, filters the Sentinel-1 dataset and vector data accordingly, and uses the <code>viz</code> function to display the results for that day.</p>
|
||||
<span id="cb4-24"><a href="#cb4-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-25"><a href="#cb4-25" aria-hidden="true" tabindex="-1"></a> <span class="co">// Use the viz function to visualize the results </span></span>
|
||||
<span id="cb4-26"><a href="#cb4-26" aria-hidden="true" tabindex="-1"></a> <span class="fu">viz</span>(aoi<span class="op">,</span> vectors<span class="op">,</span> s1Filtered<span class="op">.</span><span class="fu">max</span>()<span class="op">.</span><span class="fu">updateMask</span>(dem<span class="op">.</span><span class="fu">lte</span>(<span class="dv">0</span>)))<span class="op">;</span></span>
|
||||
<span id="cb4-27"><a href="#cb4-27" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-28"><a href="#cb4-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-29"><a href="#cb4-29" aria-hidden="true" tabindex="-1"></a> <span class="co">// Create a chart of the number of ships per day</span></span>
|
||||
<span id="cb4-30"><a href="#cb4-30" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> chart <span class="op">=</span> ui<span class="op">.</span><span class="at">Chart</span><span class="op">.</span><span class="at">feature</span></span>
|
||||
<span id="cb4-31"><a href="#cb4-31" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">byFeature</span>({</span>
|
||||
<span id="cb4-32"><a href="#cb4-32" aria-hidden="true" tabindex="-1"></a> <span class="dt">features</span><span class="op">:</span> vectors<span class="op">,</span></span>
|
||||
<span id="cb4-33"><a href="#cb4-33" aria-hidden="true" tabindex="-1"></a> <span class="dt">xProperty</span><span class="op">:</span> <span class="st">"system:time_start"</span><span class="op">,</span></span>
|
||||
<span id="cb4-34"><a href="#cb4-34" aria-hidden="true" tabindex="-1"></a> <span class="dt">yProperties</span><span class="op">:</span> [<span class="st">"count"</span>]<span class="op">,</span></span>
|
||||
<span id="cb4-35"><a href="#cb4-35" aria-hidden="true" tabindex="-1"></a> })</span>
|
||||
<span id="cb4-36"><a href="#cb4-36" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">setOptions</span>({</span>
|
||||
<span id="cb4-37"><a href="#cb4-37" aria-hidden="true" tabindex="-1"></a> <span class="dt">title</span><span class="op">:</span> <span class="st">"Daily Number of Ships in Area of Interest"</span><span class="op">,</span></span>
|
||||
<span id="cb4-38"><a href="#cb4-38" aria-hidden="true" tabindex="-1"></a> <span class="dt">vAxis</span><span class="op">:</span> { <span class="dt">title</span><span class="op">:</span> <span class="st">"Ship Count"</span> }<span class="op">,</span></span>
|
||||
<span id="cb4-39"><a href="#cb4-39" aria-hidden="true" tabindex="-1"></a> <span class="dt">explorer</span><span class="op">:</span> { <span class="dt">axis</span><span class="op">:</span> <span class="st">"horizontal"</span> }<span class="op">,</span></span>
|
||||
<span id="cb4-40"><a href="#cb4-40" aria-hidden="true" tabindex="-1"></a> <span class="dt">lineWidth</span><span class="op">:</span> <span class="dv">2</span><span class="op">,</span></span>
|
||||
<span id="cb4-41"><a href="#cb4-41" aria-hidden="true" tabindex="-1"></a> <span class="dt">series</span><span class="op">:</span> <span class="st">"Area of Interest"</span><span class="op">,</span></span>
|
||||
<span id="cb4-42"><a href="#cb4-42" aria-hidden="true" tabindex="-1"></a> })<span class="op">;</span></span>
|
||||
<span id="cb4-43"><a href="#cb4-43" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-44"><a href="#cb4-44" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-45"><a href="#cb4-45" aria-hidden="true" tabindex="-1"></a> <span class="co">// Add the chart at a fixed position, so that new charts overwrite older ones.</span></span>
|
||||
<span id="cb4-46"><a href="#cb4-46" aria-hidden="true" tabindex="-1"></a> controlPanel<span class="op">.</span><span class="fu">widgets</span>()<span class="op">.</span><span class="fu">set</span>(<span class="dv">4</span><span class="op">,</span> chart)<span class="op">;</span></span>
|
||||
<span id="cb4-47"><a href="#cb4-47" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-48"><a href="#cb4-48" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb4-49"><a href="#cb4-49" aria-hidden="true" tabindex="-1"></a> <span class="co">// Add a click handler to the chart to filter the map by day.</span></span>
|
||||
<span id="cb4-50"><a href="#cb4-50" aria-hidden="true" tabindex="-1"></a> chart<span class="op">.</span><span class="fu">onClick</span>(filterDay)<span class="op">;</span></span>
|
||||
<span id="cb4-51"><a href="#cb4-51" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>There’s one function referenced above – <code>filterDay</code>– that we haven’t defined yet. This function is called when the user clicks on the chart to analyze a particular day. It takes the date of the clicked day as an input, filters the Sentinel-1 dataset and vector data accordingly, and uses the <code>viz</code> function to display the results for that day.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">filterDay</span> (callback) {</span>
|
||||
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the date of the clicked day</span></span>
|
||||
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> date <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(callback)<span class="op">;</span></span>
|
||||
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the vector data to only include images from that day</span></span>
|
||||
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> vectorDay <span class="op">=</span> vectors<span class="op">.</span><span class="fu">filterDate</span>(date)<span class="op">;</span></span>
|
||||
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the Sentinel-1 imagery to only include images from that day</span></span>
|
||||
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> s1Day <span class="op">=</span> s1<span class="op">.</span><span class="fu">filterDate</span>(date)<span class="op">.</span><span class="fu">max</span>()<span class="op">.</span><span class="fu">updateMask</span>(dem<span class="op">.</span><span class="fu">lte</span>(<span class="dv">0</span>))<span class="op">;</span></span>
|
||||
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a> <span class="co">// Use the viz function to visualize the results</span></span>
|
||||
<span id="cb5-13"><a href="#cb5-13" aria-hidden="true" tabindex="-1"></a> <span class="fu">viz</span>(aoi<span class="op">,</span> vectorDay<span class="op">,</span> s1Day)<span class="op">;</span></span>
|
||||
<span id="cb5-14"><a href="#cb5-14" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb5-3"><a href="#cb5-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-4"><a href="#cb5-4" aria-hidden="true" tabindex="-1"></a> <span class="co">// Get the date of the clicked day</span></span>
|
||||
<span id="cb5-5"><a href="#cb5-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> date <span class="op">=</span> ee<span class="op">.</span><span class="fu">Date</span>(callback)<span class="op">;</span></span>
|
||||
<span id="cb5-6"><a href="#cb5-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-7"><a href="#cb5-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-8"><a href="#cb5-8" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the vector data to only include images from that day</span></span>
|
||||
<span id="cb5-9"><a href="#cb5-9" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> vectorDay <span class="op">=</span> vectors<span class="op">.</span><span class="fu">filterDate</span>(date)<span class="op">;</span></span>
|
||||
<span id="cb5-10"><a href="#cb5-10" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb5-11"><a href="#cb5-11" aria-hidden="true" tabindex="-1"></a> <span class="co">// Filter the Sentinel-1 imagery to only include images from that day</span></span>
|
||||
<span id="cb5-12"><a href="#cb5-12" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> s1Day <span class="op">=</span> s1<span class="op">.</span><span class="fu">filterDate</span>(date)<span class="op">.</span><span class="fu">max</span>()<span class="op">.</span><span class="fu">updateMask</span>(dem<span class="op">.</span><span class="fu">lte</span>(<span class="dv">0</span>))<span class="op">;</span></span>
|
||||
<span id="cb5-13"><a href="#cb5-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-14"><a href="#cb5-14" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb5-15"><a href="#cb5-15" aria-hidden="true" tabindex="-1"></a> <span class="co">// Use the viz function to visualize the results</span></span>
|
||||
<span id="cb5-16"><a href="#cb5-16" aria-hidden="true" tabindex="-1"></a> <span class="fu">viz</span>(aoi<span class="op">,</span> vectorDay<span class="op">,</span> s1Day)<span class="op">;</span></span>
|
||||
<span id="cb5-17"><a href="#cb5-17" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The analytical portion of the application is now complete. Now we have to build a user interface that lets us interact with the application.</p>
|
||||
</section>
|
||||
<section id="building-a-user-interface" class="level2">
|
||||
@@ -544,34 +565,39 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p>We eventually want to allow the user to draw a polygon on the map, and count the number of ships that fall within it. In order to do so, we need to set up a few functions related to the drawing tools that allow the user to do this. Among other things, we want to make sure that we’re clearing the old geometries so that we’re only ever conducting analysis inside the most recent user-drawn polygon, so we’ll need to clear the old ones. We also want to specify the type of polygon the user can draw, which for ease will be a rectangle (you could change this to the actual “polygon” type if you wanted to draw more complex geometries).</p>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> drawingTools <span class="op">=</span> <span class="bu">Map</span><span class="op">.</span><span class="fu">drawingTools</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a><span class="co">// Remove any existing layers</span></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">length</span>() <span class="op">></span> <span class="dv">0</span>) {</span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> layer <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">remove</span>(layer)<span class="op">;</span></span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a>}</span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a><span class="co">// Add a dummy layer to the drawing tools object (the Suez Canal box)</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> dummyGeometry <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">GeometryLayer</span>({</span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">geometries</span><span class="op">:</span> <span class="kw">null</span><span class="op">,</span></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a>})</span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">fromGeometry</span>(suez)</span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">setShown</span>(<span class="kw">false</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the dummy layer to the drawing tools object</span></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a>drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">add</span>(dummyGeometry)<span class="op">;</span></span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a><span class="co">// Remove any existing layers</span></span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a><span class="cf">while</span> (drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">length</span>() <span class="op">></span> <span class="dv">0</span>) {</span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> layer <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">remove</span>(layer)<span class="op">;</span></span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a>}</span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a><span class="co">// Add a dummy layer to the drawing tools object (the Suez Canal box)</span></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> dummyGeometry <span class="op">=</span> ui<span class="op">.</span><span class="at">Map</span><span class="op">.</span><span class="fu">GeometryLayer</span>({</span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">geometries</span><span class="op">:</span> <span class="kw">null</span><span class="op">,</span></span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a>})</span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">fromGeometry</span>(suez)</span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> <span class="op">.</span><span class="fu">setShown</span>(<span class="kw">false</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a function that clears existing geometries and lets the user draw a rectangle</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">drawPolygon</span>() {</span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> layers <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a> layers<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">geometries</span>()<span class="op">.</span><span class="fu">remove</span>(layers<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">geometries</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>))<span class="op">;</span></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">setShape</span>(<span class="st">"rectangle"</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">draw</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the dummy layer to the drawing tools object</span></span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a>drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">.</span><span class="fu">add</span>(dummyGeometry)<span class="op">;</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a function that clears existing geometries and lets the user draw a rectangle</span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a><span class="kw">function</span> <span class="fu">drawPolygon</span>() {</span>
|
||||
<span id="cb6-27"><a href="#cb6-27" aria-hidden="true" tabindex="-1"></a> <span class="kw">var</span> layers <span class="op">=</span> drawingTools<span class="op">.</span><span class="fu">layers</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-28"><a href="#cb6-28" aria-hidden="true" tabindex="-1"></a> layers<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">geometries</span>()<span class="op">.</span><span class="fu">remove</span>(layers<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>)<span class="op">.</span><span class="fu">geometries</span>()<span class="op">.</span><span class="fu">get</span>(<span class="dv">0</span>))<span class="op">;</span></span>
|
||||
<span id="cb6-29"><a href="#cb6-29" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">setShape</span>(<span class="st">"rectangle"</span>)<span class="op">;</span></span>
|
||||
<span id="cb6-30"><a href="#cb6-30" aria-hidden="true" tabindex="-1"></a> drawingTools<span class="op">.</span><span class="fu">draw</span>()<span class="op">;</span></span>
|
||||
<span id="cb6-31"><a href="#cb6-31" aria-hidden="true" tabindex="-1"></a>}</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
<section id="widgets" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="widgets">Widgets</h3>
|
||||
<p>The control panel will eventually contain a few different widgets that allow the user to interact with the application. We’ll start by creating a button that allows the user to draw a polygon on the map. We’ll also create a slider that allows the user to adjust the size of the ships that are detected (remember, this manipualtes the “scale” parameter in the <code>reduceToVectors</code> function used in the detection process). The slider will have an accompanying label that tells the user what it does.</p>
|
||||
<p>The control panel will eventually contain a few different widgets that allow the user to interact with the application. We’ll start by creating a button that allows the user to draw a polygon on the map. We’ll also create a slider that allows the user to adjust the size of the ships that are detected (remember, this manipulates the “scale” parameter in the <code>reduceToVectors</code> function used in the detection process). The slider will have an accompanying label that tells the user what it does.</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a button that allows the user to draw a polygon on the map</span></span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> drawButton <span class="op">=</span> ui<span class="op">.</span><span class="fu">Button</span>({</span>
|
||||
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">label</span><span class="op">:</span> <span class="st">"🔺"</span> <span class="op">+</span> <span class="st">" Draw a Polygon"</span><span class="op">,</span></span>
|
||||
@@ -579,43 +605,47 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">stretch</span><span class="op">:</span> <span class="st">"horizontal"</span> }<span class="op">,</span></span>
|
||||
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a slider that allows the user to adjust the size of the ships that are detected</span></span>
|
||||
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> scaleSlider <span class="op">=</span> ui<span class="op">.</span><span class="fu">Slider</span>({</span>
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">100</span><span class="op">,</span></span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">value</span><span class="op">:</span> <span class="dv">80</span><span class="op">,</span></span>
|
||||
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">step</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">onChange</span><span class="op">:</span> daterangeVectors<span class="op">,</span></span>
|
||||
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">width</span><span class="op">:</span> <span class="st">"70%"</span> }<span class="op">,</span></span>
|
||||
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-18"><a href="#cb7-18" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a label for the slider</span></span>
|
||||
<span id="cb7-19"><a href="#cb7-19" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> scaleLabel <span class="op">=</span> ui<span class="op">.</span><span class="fu">Label</span>(<span class="st">"Ship Size: "</span>)<span class="op">;</span></span>
|
||||
<span id="cb7-20"><a href="#cb7-20" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-21"><a href="#cb7-21" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a panel that contains the slider and its label</span></span>
|
||||
<span id="cb7-22"><a href="#cb7-22" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> scalePanel <span class="op">=</span> ui<span class="op">.</span><span class="fu">Panel</span>({</span>
|
||||
<span id="cb7-23"><a href="#cb7-23" aria-hidden="true" tabindex="-1"></a> <span class="dt">widgets</span><span class="op">:</span> [scaleLabel<span class="op">,</span> scaleSlider]<span class="op">,</span></span>
|
||||
<span id="cb7-24"><a href="#cb7-24" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">stretch</span><span class="op">:</span> <span class="st">"horizontal"</span> }<span class="op">,</span></span>
|
||||
<span id="cb7-25"><a href="#cb7-25" aria-hidden="true" tabindex="-1"></a> <span class="dt">layout</span><span class="op">:</span> ui<span class="op">.</span><span class="at">Panel</span><span class="op">.</span><span class="at">Layout</span><span class="op">.</span><span class="fu">Flow</span>(<span class="st">"horizontal"</span>)<span class="op">,</span></span>
|
||||
<span id="cb7-26"><a href="#cb7-26" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a slider that allows the user to adjust the size of the ships that are detected</span></span>
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> scaleSlider <span class="op">=</span> ui<span class="op">.</span><span class="fu">Slider</span>({</span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">min</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">max</span><span class="op">:</span> <span class="dv">100</span><span class="op">,</span></span>
|
||||
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">value</span><span class="op">:</span> <span class="dv">80</span><span class="op">,</span></span>
|
||||
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a> <span class="dt">step</span><span class="op">:</span> <span class="dv">1</span><span class="op">,</span></span>
|
||||
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a> <span class="dt">onChange</span><span class="op">:</span> daterangeVectors<span class="op">,</span></span>
|
||||
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">width</span><span class="op">:</span> <span class="st">"70%"</span> }<span class="op">,</span></span>
|
||||
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span>
|
||||
<span id="cb7-18"><a href="#cb7-18" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-19"><a href="#cb7-19" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-20"><a href="#cb7-20" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a label for the slider</span></span>
|
||||
<span id="cb7-21"><a href="#cb7-21" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> scaleLabel <span class="op">=</span> ui<span class="op">.</span><span class="fu">Label</span>(<span class="st">"Ship Size: "</span>)<span class="op">;</span></span>
|
||||
<span id="cb7-22"><a href="#cb7-22" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-23"><a href="#cb7-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-24"><a href="#cb7-24" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a panel that contains the slider and its label</span></span>
|
||||
<span id="cb7-25"><a href="#cb7-25" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> scalePanel <span class="op">=</span> ui<span class="op">.</span><span class="fu">Panel</span>({</span>
|
||||
<span id="cb7-26"><a href="#cb7-26" aria-hidden="true" tabindex="-1"></a> <span class="dt">widgets</span><span class="op">:</span> [scaleLabel<span class="op">,</span> scaleSlider]<span class="op">,</span></span>
|
||||
<span id="cb7-27"><a href="#cb7-27" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">stretch</span><span class="op">:</span> <span class="st">"horizontal"</span> }<span class="op">,</span></span>
|
||||
<span id="cb7-28"><a href="#cb7-28" aria-hidden="true" tabindex="-1"></a> <span class="dt">layout</span><span class="op">:</span> ui<span class="op">.</span><span class="at">Panel</span><span class="op">.</span><span class="at">Layout</span><span class="op">.</span><span class="fu">Flow</span>(<span class="st">"horizontal"</span>)<span class="op">,</span></span>
|
||||
<span id="cb7-29"><a href="#cb7-29" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>The last widget we’re going to define is the date slider. This widget will trigger the <code>daterangeVectors</code> function, which will filter the Sentinel-1 dataset to only include images from the selected year, and then run the detection process on the filtered dataset.</p>
|
||||
<div class="sourceCode" id="cb8"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Specify the start and end dates for the date slider</span></span>
|
||||
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> start <span class="op">=</span> <span class="st">"2014-01-01"</span><span class="op">;</span></span>
|
||||
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> now <span class="op">=</span> <span class="bu">Date</span><span class="op">.</span><span class="fu">now</span>()<span class="op">;</span></span>
|
||||
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a date slider that allows the user to select a year</span></span>
|
||||
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> dateSlider <span class="op">=</span> ui<span class="op">.</span><span class="fu">DateSlider</span>({</span>
|
||||
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">value</span><span class="op">:</span> <span class="st">"2021-03-01"</span><span class="op">,</span></span>
|
||||
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">start</span><span class="op">:</span> start<span class="op">,</span></span>
|
||||
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">end</span><span class="op">:</span> now<span class="op">,</span></span>
|
||||
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">period</span><span class="op">:</span> <span class="dv">365</span><span class="op">,</span></span>
|
||||
<span id="cb8-11"><a href="#cb8-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">onChange</span><span class="op">:</span> daterangeVectors<span class="op">,</span></span>
|
||||
<span id="cb8-12"><a href="#cb8-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">width</span><span class="op">:</span> <span class="st">"95%"</span> }<span class="op">,</span></span>
|
||||
<span id="cb8-13"><a href="#cb8-13" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a><span class="co">// Create a date slider that allows the user to select a year</span></span>
|
||||
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> dateSlider <span class="op">=</span> ui<span class="op">.</span><span class="fu">DateSlider</span>({</span>
|
||||
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a> <span class="dt">value</span><span class="op">:</span> <span class="st">"2021-03-01"</span><span class="op">,</span></span>
|
||||
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a> <span class="dt">start</span><span class="op">:</span> start<span class="op">,</span></span>
|
||||
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a> <span class="dt">end</span><span class="op">:</span> now<span class="op">,</span></span>
|
||||
<span id="cb8-11"><a href="#cb8-11" aria-hidden="true" tabindex="-1"></a> <span class="dt">period</span><span class="op">:</span> <span class="dv">365</span><span class="op">,</span></span>
|
||||
<span id="cb8-12"><a href="#cb8-12" aria-hidden="true" tabindex="-1"></a> <span class="dt">onChange</span><span class="op">:</span> daterangeVectors<span class="op">,</span></span>
|
||||
<span id="cb8-13"><a href="#cb8-13" aria-hidden="true" tabindex="-1"></a> <span class="dt">style</span><span class="op">:</span> { <span class="dt">width</span><span class="op">:</span> <span class="st">"95%"</span> }<span class="op">,</span></span>
|
||||
<span id="cb8-14"><a href="#cb8-14" aria-hidden="true" tabindex="-1"></a>})<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
</section>
|
||||
<section id="the-control-panel" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="the-control-panel">The Control Panel</h3>
|
||||
<p>Now we’re going to assemble all of the widgets we’ve just defined into one panel, alongsie some explanatory text. I’m adding a blank label to the panel as a placeholder for the chart, since it will be re-added to the panel every time the user changed the date on the date slider, the AOI, or the scale.</p>
|
||||
<p>Now we’re going to assemble all of the widgets we’ve just defined into one panel, alongside some explanatory text. I’m adding a blank label to the panel as a placeholder for the chart, since it will be re-added to the panel every time the user changes the date on the date slider, the AOI, or the scale.</p>
|
||||
<div class="sourceCode" id="cb9"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a><span class="kw">var</span> controlPanel <span class="op">=</span> ui<span class="op">.</span><span class="fu">Panel</span>({</span>
|
||||
<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">widgets</span><span class="op">:</span> [</span>
|
||||
<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a> ui<span class="op">.</span><span class="fu">Label</span>(<span class="st">"SAR Ship Detection"</span><span class="op">,</span> {</span>
|
||||
@@ -641,11 +671,13 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<div class="sourceCode" id="cb10"><pre class="sourceCode js code-with-copy"><code class="sourceCode javascript"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true" tabindex="-1"></a><span class="co">// Add the control panel to the map</span></span>
|
||||
<span id="cb10-2"><a href="#cb10-2" aria-hidden="true" tabindex="-1"></a>ui<span class="op">.</span><span class="at">root</span><span class="op">.</span><span class="fu">insert</span>(<span class="dv">0</span><span class="op">,</span>controlPanel)<span class="op">;</span></span>
|
||||
<span id="cb10-3"><a href="#cb10-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a><span class="co">// Trigger the daterangeVectors function when the user draws a polygon</span></span>
|
||||
<span id="cb10-5"><a href="#cb10-5" aria-hidden="true" tabindex="-1"></a>drawingTools<span class="op">.</span><span class="fu">onDraw</span>(ui<span class="op">.</span><span class="at">util</span><span class="op">.</span><span class="fu">debounce</span>(daterangeVectors<span class="op">,</span> <span class="dv">500</span>))<span class="op">;</span></span>
|
||||
<span id="cb10-6"><a href="#cb10-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a><span class="co">// Run the daterangeVectors function to initialize the map</span></span>
|
||||
<span id="cb10-8"><a href="#cb10-8" aria-hidden="true" tabindex="-1"></a><span class="fu">daterangeVectors</span>()<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb10-4"><a href="#cb10-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb10-5"><a href="#cb10-5" aria-hidden="true" tabindex="-1"></a><span class="co">// Trigger the daterangeVectors function when the user draws a polygon</span></span>
|
||||
<span id="cb10-6"><a href="#cb10-6" aria-hidden="true" tabindex="-1"></a>drawingTools<span class="op">.</span><span class="fu">onDraw</span>(ui<span class="op">.</span><span class="at">util</span><span class="op">.</span><span class="fu">debounce</span>(daterangeVectors<span class="op">,</span> <span class="dv">500</span>))<span class="op">;</span></span>
|
||||
<span id="cb10-7"><a href="#cb10-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb10-8"><a href="#cb10-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb10-9"><a href="#cb10-9" aria-hidden="true" tabindex="-1"></a><span class="co">// Run the daterangeVectors function to initialize the map</span></span>
|
||||
<span id="cb10-10"><a href="#cb10-10" aria-hidden="true" tabindex="-1"></a><span class="fu">daterangeVectors</span>()<span class="op">;</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>And there we have it. A fully functional, all weather, daytime/nighttime ship detection tool that doesn’t rely on AIS data. Let’s play around with it.</p>
|
||||
</section>
|
||||
</section>
|
||||
@@ -653,18 +685,18 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<h2 class="anchored" data-anchor-id="taking-it-for-a-spin">Taking it for a spin</h2>
|
||||
<section id="north-korea" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="north-korea">North Korea</h3>
|
||||
<p>In 2020, North Korea implemented one of the most severe COVID-19 lockdowns in the world including a near-total ban on <a href="https://thediplomat.com/2023/01/north-korea-likely-to-lift-pandemic-border-restrictions-in-2023/">“all cross-border exchanges, including trade, traffic, and tourism”.</a>. Measures have been so severe that country appears to have experienced a significant <a href="https://foreignpolicy.com/2022/05/16/kim-north-korea-covid-outbreak-pandemic/">famine</a>. Though there were signs that things have gradually returned to normal, information on North Korea’s economy is pretty hard to come by. Ship traffic in and out of the country’s largest port, Nampo, is probably a pretty good indicator of the country’s economic activity.</p>
|
||||
<p>But we can’t just head on down to Marine Tracker or other services that use AIS data to track ship movements. According to the <a href="https://home.treasury.gov/system/files/126/dprk_vessel_advisory_02232018.pdf">U.S. Treasury</a>, “North Korean-flagged merchant vessels have been known to intentionally disable their AIS transponders to mask their movements. This tactic, whether employed by North Korean-flagged vessels or other vessels involved in trade with North Korea, could conceal the origin or destination of cargo destined for, or originating in, North Korea.” They should know– they’re the ones imposing the sanctions that make it illegal to trade with North Korea.</p>
|
||||
<p>In 2020, North Korea implemented one of the most severe COVID-19 lockdowns in the world including a near-total ban on <a href="https://thediplomat.com/2023/01/north-korea-likely-to-lift-pandemic-border-restrictions-in-2023/">“all cross-border exchanges, including trade, traffic, and tourism”.</a>. Measures have been so severe that the country appears to have experienced a significant <a href="https://foreignpolicy.com/2022/05/16/kim-north-korea-covid-outbreak-pandemic/">famine</a>. Though there were signs that things have gradually returned to normal, information on North Korea’s economy is pretty hard to come by. Ship traffic in and out of the country’s largest port, Nampo, is probably a pretty good indicator of the country’s economic activity.</p>
|
||||
<p>But we can’t just head on down to Marine Tracker or other services that use AIS data to track ship movements. According to the <a href="https://home.treasury.gov/system/files/126/dprk_vessel_advisory_02232018.pdf">U.S. Treasury</a>, “North Korean-flagged merchant vessels have been known to intentionally disable their AIS transponders to mask their movements. This tactic, whether employed by North Korean-flagged vessels or other vessels involved in trade with North Korea, could conceal the origin or destination of cargo destined for, or originating in, North Korea.” They should know – they’re the ones imposing the sanctions that make it illegal to trade with North Korea.</p>
|
||||
<p>A New York Times <a href="https://www.nytimes.com/2019/07/16/world/asia/north-korea-luxury-goods-sanctions.html">investigation</a> tracked the maritime voyage of luxury Mercedes cars from Germany to North Korea via the Netherlands, China, Japan, South Korea, and Russia. AIS transponders were turned off at several points throughout this journey, and the investigation had to rely on satellite imagery to fill in the gaps.</p>
|
||||
<p>Though they used high resolution optical imagery to follow individual ships, we want to identify lots of ships in a large area over a long period. That would get very expensive, and automatic ship detection in optical imagery is relatively difficult. Here’s how our SAR tool fares when we draw a box in the bay of Nampo:</p>
|
||||
<p><img src="./images/ships_north_korea.jpg" class="img-fluid"></p>
|
||||
<p><img src="../images/ships_north_korea.jpg" class="img-fluid"></p>
|
||||
<p>Looking at imagery from 2021, we can see ship traffic increasing from nearly zero to around 40 ships per day.</p>
|
||||
</section>
|
||||
<section id="ukraine" class="level3">
|
||||
<h3 class="anchored" data-anchor-id="ukraine">Ukraine</h3>
|
||||
<p>Odessa is Ukraine’s largest port. Following its invasion of Ukraine in February 2022, Russia instituted a naval blockade against Ukrainian ports. The impact of this blockade is clearly visible using the tool we’ve just built:</p>
|
||||
<p><img src="./images/ships_ukraine.jpg" class="img-fluid"></p>
|
||||
<p>The daily number of ships detected in the port of Odessa dropped from 40-50 to 0-5 following the invasion, and remained near zero until the blockade was lifted in September 2022.</p>
|
||||
<p><img src="../images/ships_ukraine.jpg" class="img-fluid"></p>
|
||||
<p>The daily number of ships detected in the port of Odessa dropped from 40 to 50 to 0 to 5 following the invasion, and remained near zero until the blockade was lifted in September 2022.</p>
|
||||
|
||||
|
||||
</section>
|
||||
@@ -922,13 +954,13 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./refineries.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text">Refinery Identification</span>
|
||||
<a href="../chapters/C3_Blast.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Blast Damage Assessment</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
<a href="./blast.html" class="pagination-link">
|
||||
<span class="nav-page-text">Blast Damage Assessment</span> <i class="bi bi-arrow-right-short"></i>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="pagination-link">
|
||||
<span class="nav-page-text"><span class="chapter-title">Object Detection</span></span> <i class="bi bi-arrow-right-short"></i>
|
||||
</a>
|
||||
</div>
|
||||
</nav>
|
||||
@@ -7,7 +7,7 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
|
||||
|
||||
|
||||
<title>Remote Sensing for OSINT - Object Detection</title>
|
||||
<title>Remote Sensing for OSINT - 11 Object Detection</title>
|
||||
<style>
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
@@ -86,28 +86,28 @@ code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warni
|
||||
</style>
|
||||
|
||||
|
||||
<script src="site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="./">
|
||||
<link href="./blast.html" rel="prev">
|
||||
<link href="./favicon.ico" rel="icon">
|
||||
<script src="site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="site_libs/quarto-contrib/videojs/video.min.js"></script>
|
||||
<link href="site_libs/quarto-contrib/videojs/video-js.css" rel="stylesheet">
|
||||
<script src="../site_libs/quarto-nav/quarto-nav.js"></script>
|
||||
<script src="../site_libs/quarto-nav/headroom.min.js"></script>
|
||||
<script src="../site_libs/clipboard/clipboard.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/autocomplete.umd.js"></script>
|
||||
<script src="../site_libs/quarto-search/fuse.min.js"></script>
|
||||
<script src="../site_libs/quarto-search/quarto-search.js"></script>
|
||||
<meta name="quarto:offset" content="../">
|
||||
<link href="../chapters/C4_Ships.html" rel="prev">
|
||||
<link href="../favicon.ico" rel="icon">
|
||||
<script src="../site_libs/quarto-html/quarto.js"></script>
|
||||
<script src="../site_libs/quarto-html/popper.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/tippy.umd.min.js"></script>
|
||||
<script src="../site_libs/quarto-html/anchor.min.js"></script>
|
||||
<link href="../site_libs/quarto-html/tippy.css" rel="stylesheet">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-text-highlighting-styles">
|
||||
<link href="../site_libs/quarto-html/quarto-syntax-highlighting-dark.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-text-highlighting-styles">
|
||||
<script src="../site_libs/bootstrap/bootstrap.min.js"></script>
|
||||
<link href="../site_libs/bootstrap/bootstrap-icons.css" rel="stylesheet">
|
||||
<link href="../site_libs/bootstrap/bootstrap.min.css" rel="stylesheet" class="quarto-color-scheme" id="quarto-bootstrap" data-mode="light">
|
||||
<link href="../site_libs/bootstrap/bootstrap-dark.min.css" rel="stylesheet" class="quarto-color-scheme quarto-color-alternate" id="quarto-bootstrap" data-mode="dark">
|
||||
<script src="../site_libs/quarto-contrib/videojs/video.min.js"></script>
|
||||
<link href="../site_libs/quarto-contrib/videojs/video-js.css" rel="stylesheet">
|
||||
<script id="quarto-search-options" type="application/json">{
|
||||
"location": "sidebar",
|
||||
"copy-button": false,
|
||||
@@ -148,7 +148,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<header id="quarto-header" class="headroom fixed-top">
|
||||
<nav class="quarto-secondary-nav" data-bs-toggle="collapse" data-bs-target="#quarto-sidebar" aria-controls="quarto-sidebar" aria-expanded="false" aria-label="Toggle sidebar navigation" onclick="if (window.quartoToggleHeadroom) { window.quartoToggleHeadroom(); }">
|
||||
<div class="container-fluid d-flex justify-content-between">
|
||||
<h1 class="quarto-secondary-nav-title">Object Detection</h1>
|
||||
<h1 class="quarto-secondary-nav-title"><span class="chapter-title">Object Detection</span></h1>
|
||||
<button type="button" class="quarto-btn-toggle btn" aria-label="Show secondary navigation">
|
||||
<i class="bi bi-chevron-right"></i>
|
||||
</button>
|
||||
@@ -160,24 +160,24 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<!-- sidebar -->
|
||||
<nav id="quarto-sidebar" class="sidebar collapse sidebar-navigation floating overflow-auto">
|
||||
<div class="pt-lg-2 mt-2 text-left sidebar-header sidebar-header-stacked">
|
||||
<a href="./index.html" class="sidebar-logo-link">
|
||||
<img src="./logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
<a href="../index.html" class="sidebar-logo-link">
|
||||
<img src="../images/logo_white.png" alt="" class="sidebar-logo py-0 d-lg-inline d-none">
|
||||
</a>
|
||||
<div class="sidebar-title mb-0 py-0">
|
||||
<a href="./">Remote Sensing for OSINT</a>
|
||||
<a href="../">Remote Sensing for OSINT</a>
|
||||
<div class="sidebar-tools-main tools-wide">
|
||||
<a href="https://github.com/oballinger/GEE_OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="https://github.com/oballinger/RS4OSINT/" title="Source Code" class="sidebar-tool px-1"><i class="bi bi-github"></i></a>
|
||||
<a href="" title="Download" id="sidebar-tool-dropdown-0" class="sidebar-tool dropdown-toggle px-1" data-bs-toggle="dropdown" aria-expanded="false"><i class="bi bi-download"></i></a>
|
||||
<ul class="dropdown-menu" aria-labelledby="sidebar-tool-dropdown-0">
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.pdf">
|
||||
<i class="bi bi-bi-file-pdf pe-1"></i>
|
||||
Download PDF
|
||||
</a>
|
||||
</li>
|
||||
<li>
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="./Remote-Sensing-
|
||||
<a class="dropdown-item sidebar-tools-main-item" href="../Remote-Sensing-
|
||||
-for-OSINT.epub">
|
||||
<i class="bi bi-bi-journal pe-1"></i>
|
||||
Download ePub
|
||||
@@ -220,17 +220,17 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-1" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
<a href="../index.html" class="sidebar-item-text sidebar-link">Overview</a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch1.html" class="sidebar-item-text sidebar-link">Remote Sensing</a>
|
||||
<a href="../chapters/A2_Remote_Sensing.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Remote Sensing</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ch2.html" class="sidebar-item-text sidebar-link">Data Acquisition</a>
|
||||
<a href="../chapters/A3_Data_Acquisition.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Data Acquisition</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -245,22 +245,22 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-2" class="collapse list-unstyled sidebar-section depth1 ">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F1.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">1</span> <span class="chapter-title">Getting Started</span></a>
|
||||
<a href="../chapters/B1_Getting_Started.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Getting Started</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F2.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">2</span> <span class="chapter-title">Interpreting Images</span></a>
|
||||
<a href="../chapters/B2_Interpreting_Images.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Interpreting Images</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F4.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">3</span> <span class="chapter-title">Image Series</span></a>
|
||||
<a href="../chapters/B3_Image_Series.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Image Series</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./F5.html" class="sidebar-item-text sidebar-link"><span class="chapter-number">4</span> <span class="chapter-title">Vectors and Tables</span></a>
|
||||
<a href="../chapters/B4_Vectors_Tables.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Vectors and Tables</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -275,27 +275,27 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<ul id="quarto-sidebar-section-3" class="collapse list-unstyled sidebar-section depth1 show">
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./lights.html" class="sidebar-item-text sidebar-link">War at Night</a>
|
||||
<a href="../chapters/C1_Lights.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">War at Night</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./refineries.html" class="sidebar-item-text sidebar-link">Refinery Identification</a>
|
||||
<a href="../chapters/C2_Refineries.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Refinery Identification</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./ships.html" class="sidebar-item-text sidebar-link">Ship Detection</a>
|
||||
<a href="../chapters/C3_Blast.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Blast Damage Assessment</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./blast.html" class="sidebar-item-text sidebar-link">Blast Damage Assessment</a>
|
||||
<a href="../chapters/C4_Ships.html" class="sidebar-item-text sidebar-link"><span class="chapter-title">Ship Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
<li class="sidebar-item">
|
||||
<div class="sidebar-item-container">
|
||||
<a href="./object_detection.html" class="sidebar-item-text sidebar-link active">Object Detection</a>
|
||||
<a href="../chapters/C5_Object_Detection.html" class="sidebar-item-text sidebar-link active"><span class="chapter-title">Object Detection</span></a>
|
||||
</div>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -323,14 +323,14 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<li><a href="#loading-the-input-imagery" id="toc-loading-the-input-imagery" class="nav-link" data-scroll-target="#loading-the-input-imagery">2. Loading the input imagery</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/GEE_OSINT/edit/main/object_detection.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
<div class="toc-actions"><div><i class="bi bi-github"></i></div><div class="action-links"><p><a href="https://github.com/oballinger/RS4OSINT/edit/main/chapters/C5_Object_Detection.qmd" class="toc-action">Edit this page</a></p></div></div></nav>
|
||||
</div>
|
||||
<!-- main -->
|
||||
<main class="content page-columns page-full" id="quarto-document-content">
|
||||
|
||||
<header id="title-block-header" class="quarto-title-block default">
|
||||
<div class="quarto-title">
|
||||
<h1 class="title d-none d-lg-block">Object Detection</h1>
|
||||
<h1 class="title d-none d-lg-block"><span class="chapter-title">Object Detection</span></h1>
|
||||
</div>
|
||||
|
||||
|
||||
@@ -360,7 +360,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</div>
|
||||
<section id="object-detection-in-satellite-imagery" class="level2">
|
||||
<h2 class="anchored" data-anchor-id="object-detection-in-satellite-imagery">Object Detection in Satellite Imagery</h2>
|
||||
<p>Object detction in satellite imagery has a variety of useful applications.</p>
|
||||
<p>Object detection in satellite imagery has a variety of useful applications.</p>
|
||||
<p>There’s the needle-in-a-haystack problem of needing to monitor a large area for a small number of objects. Immediately prior to the invasion of Ukraine, for example, a number of articles emerged showing Russian military vehicles and equipment popping up in small clearings in the forest near the border with Ukraine. Many of these deployments were spotted by painstakingly combing through high resolution satellite imagery, looking for things that look like trucks. One problem with this approach is that you need to know roughly where to look. The second, and more serious problem, is that you need to be on the lookout in the first place. Object detection, applied to satellite imagery, can automatically comb through vast areas and identify objects of interest. If planes and trucks start showing up in unexpected places, you’ll know about it.</p>
|
||||
<p>Perhaps you’re not monitoring that large of an area, but you want frequent updates about what’s going on. What sorts of objects (planes, trucks, cars, etc.) are present? How many of each? Where are they located? Instead of having to manually look through new imagery as it becomes available, you could have a model automatically analyze new collections and output a summary.</p>
|
||||
<section id="yolov5" class="level3">
|
||||
@@ -382,20 +382,21 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="op">%</span>pip install <span class="op">-</span>qr requirements.txt <span class="co"># install dependencies</span></span>
|
||||
<span id="cb1-4"><a href="#cb1-4" aria-hidden="true" tabindex="-1"></a><span class="op">%</span>pip install <span class="op">-</span>q roboflow <span class="co"># install roboflow</span></span>
|
||||
<span id="cb1-5"><a href="#cb1-5" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> torch <span class="co"># install pytorch</span></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> os <span class="co"># for os related operations</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> IPython.display <span class="im">import</span> Image, clear_output <span class="co"># to display images</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Once we’ve downloaded the YOLOv5 repository, we’ll need to download a dataset of labelled satellite imagery. For this example, we’re going to stick with ship detection as our use case, but expand upon it. We want to be able to distinguish between different types of ships, and we want to use freely-available satellite imagery.</p>
|
||||
<span id="cb1-6"><a href="#cb1-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb1-7"><a href="#cb1-7" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> torch <span class="co"># install pytorch</span></span>
|
||||
<span id="cb1-8"><a href="#cb1-8" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> os <span class="co"># for os related operations</span></span>
|
||||
<span id="cb1-9"><a href="#cb1-9" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> IPython.display <span class="im">import</span> Image, clear_output <span class="co"># to display images</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Once we’ve downloaded the YOLOv5 repository, we’ll need to download a dataset of labeled satellite imagery. For this example, we’re going to stick with ship detection as our use case, but expand upon it. We want to be able to distinguish between different types of ships, and we want to use freely-available satellite imagery.</p>
|
||||
<p>To that end, we’ll be using <a href="https://universe.roboflow.com/ibl-huczk/ships-2fvbx">this dataset</a>, which contains 3400 labeled images taken from Sentinel-2 (10m/px) and PlanetScope (3m/px) satellites. Ships in these images are labeled by drawing an outline around them:</p>
|
||||
<p><img src="images/sample_training_ships.jpg" class="img-fluid"></p>
|
||||
<p>The image above shows three ships and what is known as an STS– a “Ship-To-Ship” transfer– which is when a ship is transferring cargo to another ship. There are a total of seven classes of ship in this dataset:</p>
|
||||
<p>The image above shows three ships and what is known as an STS – a “Ship-To-Ship” transfer – which is when a ship is transferring cargo to another ship. There are a total of seven classes of ship in this dataset:</p>
|
||||
<p><img src="images/label_freq.jpg" class="img-fluid"></p>
|
||||
<p>This dataset can be downloaded directly from Roboflow using the following code:</p>
|
||||
<div class="sourceCode" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> roboflow <span class="im">import</span> Roboflow</span>
|
||||
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a>rf <span class="op">=</span> Roboflow(api_key<span class="op">=</span><span class="st">"<YOUR API KEY>"</span>)</span>
|
||||
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a>project <span class="op">=</span> rf.workspace(<span class="st">'ibl-huczk'</span>).project(<span class="st">"ships-2fvbx"</span>)</span>
|
||||
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a>dataset <span class="op">=</span> project.version(<span class="st">"1"</span>).download(<span class="st">"yolov5"</span>)</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>You’ll need to get your own API key from Roboflow, which you can do <a href="https://app.roboflow.com/account/api">here</a>, and insert it in the second line of code. Roboflow is a platform for managing and training deep learning models on custom datasets. It’s free to use for up to 3 projects, and hosts a large number of datasets that you can use to train your models. To use a different dataset, you can simply change the project name and version number in the second and third lines of code.</p>
|
||||
<p>You’ll need to get your own API key from Roboflow, which you can do <a href="https://app.roboflow.com/account/api">here</a>, and insert it in the second line of code. Roboflow is a platform for managing and training deep learning models on custom datasets. It’s free to use for up to three projects, and hosts a large number of datasets that you can use to train your models. To use a different dataset, you can simply change the project name and version number in the second and third lines of code.</p>
|
||||
<p>Finally, we can train our YOLOv5 model on the dataset we just downloaded in just one line of code:</p>
|
||||
<div class="sourceCode" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="op">!</span>python train.py <span class="op">--</span>data {dataset.location}<span class="op">/</span>data.yaml <span class="op">--</span>batch <span class="dv">32</span> <span class="op">--</span>cache</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>This should take about an hour.</p>
|
||||
@@ -407,7 +408,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</iframe>
|
||||
</div>
|
||||
<p>One metric in particular, <strong>mAP 0.5</strong>, is a good indicator of how well our model is performing. We can see it increasing rapidly at first, and then leveling off after around 30 epochs of training. The rest of this subsection will explain what exactly the mAP 0.5 value represents in this context. If you’re interested in training your own model at some point, the rest of this subsection will be of interest. If you’re just interested in deploying a pre-trained model, you can skip ahead to the next subsection.</p>
|
||||
<p>In the past when we’ve worked on machine learning projects (for example in the makeshift refinery identifion tutorial), our training and validation data was a set of points– geographic coordinates– which we labeled as either being a refinery or not. Calculating the accuracy of that model was fairly straightforward, since predictions were either true positives, true negatives, false positives, or false negatives.</p>
|
||||
<p>In the past when we’ve worked on machine learning projects (for example in the makeshift refinery identifion tutorial), our training and validation data was a set of points – geographic coordinates – which we labeled as either being a refinery or not. Calculating the accuracy of that model was fairly straightforward, since predictions were either true positives, true negatives, false positives or false negatives.</p>
|
||||
<p>This is slightly more complicated for object detection. We’re not going pixel-by-pixel and trying to say “this is a ship” or “this is not a ship.” Instead, we’re looking at a larger image, and trying to draw boxes around the ships. The problem is that there are many ways to draw a box around a ship. The image below shows the labels used in our training data to indicate the location of ships.</p>
|
||||
<p><img src="images/val_batch0_labels.jpg" class="img-fluid"> <img src="images/val_batch0_pred.jpg" class="img-fluid"></p>
|
||||
<p>The predicted bounding boxes are very close to the actual bounding boxes, but they’re not exactly the same. The first step in evaluating the performance of our model is to determine how close the predicted boxes are to the actual boxes. We can do this by calculating the <strong>intersection over union</strong> (IoU) of the predicted and actual boxes. This is essentially a measure of how much overlap there is between the the predicted and actual boxes:</p>
|
||||
@@ -430,7 +431,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</div>
|
||||
<p>Starting from the top left corner, we set a very high confidence threshold: precision is 1, meaning that every box we draw is a ship, but recall is near 0 meaning that we’re not detecting any ships. As we lower the confidence threshold, we start to detect more ships, but we also start to draw boxes around things that aren’t ships. Towards the middle of the curve, we’re detecting most of the ships, but we’re also drawing boxes around a lot of false positives. Towards the bottom right corner, we’re detecting all the ships, but we’re also generating lots of false positives.</p>
|
||||
<p>The goal is to find the point on the curve where precision and recall are both high; the closer the peak of our curve is to the top right corner, the better. A perfect model would touch the top right corner: it would have precision of 1 and recall of 1, detecting all of the ships without making any false positives. The area under this curve is called the <strong>Average Precision</strong> (AP), and is a measure of how close the curve is to the top right corner. The perfect model would have an AP of 1.</p>
|
||||
<p>Some of classes have a very high AP– the value for the Aircraft Carrier class is 0.995, which is very high (though this could be down to the fact that we have a relatively small number of images with aircraft carriers in them). Ship-To-Ship (STS) transfer operations also have a high AP, at 0.951. However, other classes– notably the “Ship” class– have a low AP. This may be because the “Ship” class is a catch-all for any ship that doesn’t fit into one of the other classes, so it encompasses lots of weird looking ships.</p>
|
||||
<p>Some classes have a very high AP – the value for the Aircraft Carrier class is 0.995, which is very high (though this could be down to the fact that we have a relatively small number of images with aircraft carriers in them). Ship-To-Ship (STS) transfer operations also have a high AP, at 0.951. However, other classes – notably the “Ship” class – have a low AP. This may be because the “Ship” class is a catch-all for any ship that doesn’t fit into one of the other classes, so it encompasses lots of weird looking ships.</p>
|
||||
<p>Finally, the <strong>mean Average Precision</strong> (mAP) is the average of the AP for each class, shown as the thick blue line above. Remember, all of this is premised on using a 0.5 threshold in the overlap (IoU) between our predicted boxes and the labels, which is why the final metric is called <strong>mAP 0.5</strong>. The mAP 0.5 for our model is 0.775, which is pretty good.</p>
|
||||
<p>This number is very useful when training a model in several different ways using the same dataset, in order to select the best performing one. It’s not that useful for comparing models trained on different datasets, since the mAP 0.5 is dependent on the number of classes in the dataset and the nature of those classes. For example, in the next section we’ll be using a different model trained on the DOTA dataset which has a mAP 0.5 of around 0.68, largely due to the fact that it has around twice as many classes and many of them are similar to each other.</p>
|
||||
</section>
|
||||
@@ -470,7 +471,7 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
<p>Once we’ve done this, we’ll also need to log in to Google Earth Engine using its Python API in order to access the satellite imagery. Running these two lines of code will generate a prompt with instructions; you have to click the link, confirm that you give the notebook permission to access your Earth Engine account, and paste the authentication code in the provided dialogue box.</p>
|
||||
<div class="sourceCode" id="cb5"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true" tabindex="-1"></a>ee.Authenticate()</span>
|
||||
<span id="cb5-2"><a href="#cb5-2" aria-hidden="true" tabindex="-1"></a>ee.Initialize()</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Great– now we can load high resolution imagery from the National Agriculture Imagery Program (NAIP) and create an interactive map. For this example, I’m centering the map on the <a href="https://en.wikipedia.org/wiki/309th_Aerospace_Maintenance_and_Regeneration_Group">Davis-Monthan Airplane Boneyard</a>. This is where the airforce retires and restores aircraft, so it will have lots of airplanes of different kinds for us to identify.</p>
|
||||
<p>Great, now we can load high resolution imagery from the National Agriculture Imagery Program (NAIP) and create an interactive map. For this example, I’m centering the map on the <a href="https://en.wikipedia.org/wiki/309th_Aerospace_Maintenance_and_Regeneration_Group">Davis-Monthan Airplane Boneyard</a>. This is where the US Air force retires and restores aircraft, so it will have lots of airplanes of different kinds for us to identify.</p>
|
||||
<p>First, we want to define a function called <code>detect</code> that will accept four arguments:</p>
|
||||
<ul>
|
||||
<li><code>input</code>: the satellite imagery we want to analyze.</li>
|
||||
@@ -480,62 +481,73 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</ul>
|
||||
<div class="sourceCode" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> detect(<span class="bu">input</span>, visParams, weight, labels<span class="op">=</span><span class="va">True</span>):</span>
|
||||
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> <span class="co"># Get the AOI from the map</span></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> aoi <span class="op">=</span> ee.FeatureCollection(Map.draw_features)</span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> mapScale<span class="op">=</span>Map.getScale()</span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a> <span class="co"># Visualize the raster in Earth Engine and get a download URL</span></span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a> image_url<span class="op">=</span><span class="bu">input</span>.visualize(bands<span class="op">=</span>visParams[<span class="st">'bands'</span>], <span class="bu">max</span><span class="op">=</span>visParams[<span class="st">'max'</span>]).getThumbURL({<span class="st">"region"</span>:aoi.geometry(), <span class="st">'scale'</span>:mapScale})</span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> <span class="co"># Load the image into a PIL image</span></span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a> response <span class="op">=</span> requests.get(image_url)</span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a> img <span class="op">=</span> Image.<span class="bu">open</span>(BytesIO(response.content))</span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> <span class="co"># Load the model</span></span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> model <span class="op">=</span>torch.hub.load(<span class="st">'.'</span>,<span class="st">'custom'</span>, path<span class="op">=</span><span class="st">'weights/</span><span class="sc">{}</span><span class="st">.pt'</span>.<span class="bu">format</span>(weight),source<span class="op">=</span><span class="st">'local'</span>,_verbose<span class="op">=</span><span class="va">False</span>)</span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a> <span class="co"># Run inference</span></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> results <span class="op">=</span> model(img)</span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> <span class="co"># Count the number of detections</span></span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> counts<span class="op">=</span>pd.DataFrame(results.pandas().xyxy[<span class="dv">0</span>].groupby(<span class="st">'name'</span>).size()).reset_index().rename(columns<span class="op">=</span>{<span class="dv">0</span>:<span class="st">'count'</span>,<span class="st">'name'</span>:<span class="st">'detected'</span>}).set_index(<span class="st">'count'</span>)</span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a> <span class="co"># Display the results</span></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a> results.show(labels<span class="op">=</span>labels)</span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a> <span class="co"># Print the number of detections and the date of the image</span></span>
|
||||
<span id="cb6-27"><a href="#cb6-27" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(ee.Date(<span class="bu">input</span>.get(<span class="st">'system:time_start'</span>)).<span class="bu">format</span>(<span class="st">"dd-MM-yyyy"</span>).getInfo())</span>
|
||||
<span id="cb6-28"><a href="#cb6-28" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(counts)</span>
|
||||
<span id="cb6-29"><a href="#cb6-29" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb6-30"><a href="#cb6-30" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> counts</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-4"><a href="#cb6-4" aria-hidden="true" tabindex="-1"></a> <span class="co"># Get the AOI from the map</span></span>
|
||||
<span id="cb6-5"><a href="#cb6-5" aria-hidden="true" tabindex="-1"></a> aoi <span class="op">=</span> ee.FeatureCollection(Map.draw_features)</span>
|
||||
<span id="cb6-6"><a href="#cb6-6" aria-hidden="true" tabindex="-1"></a> mapScale<span class="op">=</span>Map.getScale()</span>
|
||||
<span id="cb6-7"><a href="#cb6-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-8"><a href="#cb6-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-9"><a href="#cb6-9" aria-hidden="true" tabindex="-1"></a> <span class="co"># Visualize the raster in Earth Engine and get a download URL</span></span>
|
||||
<span id="cb6-10"><a href="#cb6-10" aria-hidden="true" tabindex="-1"></a> image_url<span class="op">=</span><span class="bu">input</span>.visualize(bands<span class="op">=</span>visParams[<span class="st">'bands'</span>], <span class="bu">max</span><span class="op">=</span>visParams[<span class="st">'max'</span>]).getThumbURL({<span class="st">"region"</span>:aoi.geometry(), <span class="st">'scale'</span>:mapScale})</span>
|
||||
<span id="cb6-11"><a href="#cb6-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-12"><a href="#cb6-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-13"><a href="#cb6-13" aria-hidden="true" tabindex="-1"></a> <span class="co"># Load the image into a PIL image</span></span>
|
||||
<span id="cb6-14"><a href="#cb6-14" aria-hidden="true" tabindex="-1"></a> response <span class="op">=</span> requests.get(image_url)</span>
|
||||
<span id="cb6-15"><a href="#cb6-15" aria-hidden="true" tabindex="-1"></a> img <span class="op">=</span> Image.<span class="bu">open</span>(BytesIO(response.content))</span>
|
||||
<span id="cb6-16"><a href="#cb6-16" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-17"><a href="#cb6-17" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-18"><a href="#cb6-18" aria-hidden="true" tabindex="-1"></a> <span class="co"># Load the model</span></span>
|
||||
<span id="cb6-19"><a href="#cb6-19" aria-hidden="true" tabindex="-1"></a> model <span class="op">=</span>torch.hub.load(<span class="st">'.'</span>,<span class="st">'custom'</span>, path<span class="op">=</span><span class="st">'weights/</span><span class="sc">{}</span><span class="st">.pt'</span>.<span class="bu">format</span>(weight),source<span class="op">=</span><span class="st">'local'</span>,_verbose<span class="op">=</span><span class="va">False</span>)</span>
|
||||
<span id="cb6-20"><a href="#cb6-20" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb6-21"><a href="#cb6-21" aria-hidden="true" tabindex="-1"></a> <span class="co"># Run inference</span></span>
|
||||
<span id="cb6-22"><a href="#cb6-22" aria-hidden="true" tabindex="-1"></a> results <span class="op">=</span> model(img)</span>
|
||||
<span id="cb6-23"><a href="#cb6-23" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-24"><a href="#cb6-24" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-25"><a href="#cb6-25" aria-hidden="true" tabindex="-1"></a> <span class="co"># Count the number of detections</span></span>
|
||||
<span id="cb6-26"><a href="#cb6-26" aria-hidden="true" tabindex="-1"></a> counts<span class="op">=</span>pd.DataFrame(results.pandas().xyxy[<span class="dv">0</span>].groupby(<span class="st">'name'</span>).size()).reset_index().rename(columns<span class="op">=</span>{<span class="dv">0</span>:<span class="st">'count'</span>,<span class="st">'name'</span>:<span class="st">'detected'</span>}).set_index(<span class="st">'count'</span>)</span>
|
||||
<span id="cb6-27"><a href="#cb6-27" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-28"><a href="#cb6-28" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-29"><a href="#cb6-29" aria-hidden="true" tabindex="-1"></a> <span class="co"># Display the results</span></span>
|
||||
<span id="cb6-30"><a href="#cb6-30" aria-hidden="true" tabindex="-1"></a> results.show(labels<span class="op">=</span>labels)</span>
|
||||
<span id="cb6-31"><a href="#cb6-31" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-32"><a href="#cb6-32" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb6-33"><a href="#cb6-33" aria-hidden="true" tabindex="-1"></a> <span class="co"># Print the number of detections and the date of the image</span></span>
|
||||
<span id="cb6-34"><a href="#cb6-34" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(ee.Date(<span class="bu">input</span>.get(<span class="st">'system:time_start'</span>)).<span class="bu">format</span>(<span class="st">"dd-MM-yyyy"</span>).getInfo())</span>
|
||||
<span id="cb6-35"><a href="#cb6-35" aria-hidden="true" tabindex="-1"></a> <span class="bu">print</span>(counts)</span>
|
||||
<span id="cb6-36"><a href="#cb6-36" aria-hidden="true" tabindex="-1"></a> </span>
|
||||
<span id="cb6-37"><a href="#cb6-37" aria-hidden="true" tabindex="-1"></a> <span class="cf">return</span> counts</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Now, we can load the NAIP imagery and create an interactive map.</p>
|
||||
<div class="sourceCode" id="cb7"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true" tabindex="-1"></a><span class="co"># load the past 10 years of NAIP imagery</span></span>
|
||||
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true" tabindex="-1"></a>naip <span class="op">=</span> ee.ImageCollection(<span class="st">'USDA/NAIP/DOQQ'</span>).<span class="bu">filter</span>(ee.Filter.date(<span class="st">'2012-01-01'</span>, <span class="st">'2022-01-01'</span>))</span>
|
||||
<span id="cb7-3"><a href="#cb7-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a><span class="co"># set some thresholds</span></span>
|
||||
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a>trueColorVis <span class="op">=</span> {</span>
|
||||
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a> <span class="st">'bands'</span>:[<span class="st">'R'</span>, <span class="st">'G'</span>, <span class="st">'B'</span>],</span>
|
||||
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> <span class="st">'min'</span>: <span class="dv">0</span>,</span>
|
||||
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> <span class="st">'max'</span>: <span class="dv">300</span>,</span>
|
||||
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span>
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a><span class="co"># initialize our map</span></span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a>Map <span class="op">=</span> geemap.Map()</span>
|
||||
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a>Map.setCenter(<span class="op">-</span><span class="fl">110.84</span>,<span class="fl">32.16</span>,<span class="dv">17</span>)</span>
|
||||
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a>Map.addLayer(naip.first(), trueColorVis, <span class="st">"naip"</span>)</span>
|
||||
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a>Map</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb7-4"><a href="#cb7-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-5"><a href="#cb7-5" aria-hidden="true" tabindex="-1"></a><span class="co"># set some thresholds</span></span>
|
||||
<span id="cb7-6"><a href="#cb7-6" aria-hidden="true" tabindex="-1"></a>trueColorVis <span class="op">=</span> {</span>
|
||||
<span id="cb7-7"><a href="#cb7-7" aria-hidden="true" tabindex="-1"></a> <span class="st">'bands'</span>:[<span class="st">'R'</span>, <span class="st">'G'</span>, <span class="st">'B'</span>],</span>
|
||||
<span id="cb7-8"><a href="#cb7-8" aria-hidden="true" tabindex="-1"></a> <span class="st">'min'</span>: <span class="dv">0</span>,</span>
|
||||
<span id="cb7-9"><a href="#cb7-9" aria-hidden="true" tabindex="-1"></a> <span class="st">'max'</span>: <span class="dv">300</span>,</span>
|
||||
<span id="cb7-10"><a href="#cb7-10" aria-hidden="true" tabindex="-1"></a>}<span class="op">;</span></span>
|
||||
<span id="cb7-11"><a href="#cb7-11" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-12"><a href="#cb7-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb7-13"><a href="#cb7-13" aria-hidden="true" tabindex="-1"></a><span class="co"># initialize our map</span></span>
|
||||
<span id="cb7-14"><a href="#cb7-14" aria-hidden="true" tabindex="-1"></a>Map <span class="op">=</span> geemap.Map()</span>
|
||||
<span id="cb7-15"><a href="#cb7-15" aria-hidden="true" tabindex="-1"></a>Map.setCenter(<span class="op">-</span><span class="fl">110.84</span>,<span class="fl">32.16</span>,<span class="dv">17</span>)</span>
|
||||
<span id="cb7-16"><a href="#cb7-16" aria-hidden="true" tabindex="-1"></a>Map.addLayer(naip.first(), trueColorVis, <span class="st">"naip"</span>)</span>
|
||||
<span id="cb7-17"><a href="#cb7-17" aria-hidden="true" tabindex="-1"></a>Map</span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>This will generate a small map with some drawing tools on the left side. We can use these tools to draw a polygon around the area we want to analyze. Use the drawing tools to draw a rectangle around an area of interest.</p>
|
||||
<p>Finally, we can run the detection on the imagery. We’ll do this by iterating through the collection of images, and running the <code>detect</code> function on each one. We’ll also store the results in a dataframe so we can analyze them later.</p>
|
||||
<div class="sourceCode" id="cb8"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Get the polygon we just drew on the map </span></span>
|
||||
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a>aoi<span class="op">=</span>ee.FeatureCollection(Map.draw_features)</span>
|
||||
<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a><span class="co"># Get a list of all the images in the collection</span></span>
|
||||
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a>naip_list<span class="op">=</span>naip.filterBounds(aoi).toList(naip.size())</span>
|
||||
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a><span class="co"># Iterate through the list of images and run detection on each one</span></span>
|
||||
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> num <span class="kw">in</span> <span class="bu">range</span>(<span class="dv">0</span>,(img_list.size()).getInfo()):</span>
|
||||
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a> detect(ee.Image(naip_list.get(num)), trueColorVis,<span class="st">'general'</span>,labels<span class="op">=</span><span class="va">False</span>)</span>
|
||||
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a> df<span class="op">=</span>df.append(detection) <span class="co"># store the results in a dataframe</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<span id="cb8-4"><a href="#cb8-4" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-5"><a href="#cb8-5" aria-hidden="true" tabindex="-1"></a><span class="co"># Get a list of all the images in the collection</span></span>
|
||||
<span id="cb8-6"><a href="#cb8-6" aria-hidden="true" tabindex="-1"></a>naip_list<span class="op">=</span>naip.filterBounds(aoi).toList(naip.size())</span>
|
||||
<span id="cb8-7"><a href="#cb8-7" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-8"><a href="#cb8-8" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb8-9"><a href="#cb8-9" aria-hidden="true" tabindex="-1"></a><span class="co"># Iterate through the list of images and run detection on each one</span></span>
|
||||
<span id="cb8-10"><a href="#cb8-10" aria-hidden="true" tabindex="-1"></a><span class="cf">for</span> num <span class="kw">in</span> <span class="bu">range</span>(<span class="dv">0</span>,(img_list.size()).getInfo()):</span>
|
||||
<span id="cb8-11"><a href="#cb8-11" aria-hidden="true" tabindex="-1"></a> detect(ee.Image(naip_list.get(num)), trueColorVis,<span class="st">'general'</span>,labels<span class="op">=</span><span class="va">False</span>)</span>
|
||||
<span id="cb8-12"><a href="#cb8-12" aria-hidden="true" tabindex="-1"></a> df<span class="op">=</span>df.append(detection) <span class="co"># store the results in a dataframe</span></span></code><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></pre></div>
|
||||
<p>Below is the result of the detection on the latest image in the collection:</p>
|
||||
<div class="column-screen">
|
||||
<div class="quarto-figure quarto-figure-center">
|
||||
@@ -545,9 +557,10 @@ gtag('config', 'G-RK9ZLZQ6GL', { 'anonymize_ip': true});
|
||||
</figure>
|
||||
</div>
|
||||
</div>
|
||||
<p>This image shows a remarkable degree of accuracy being achieved by our model. Inference took just 822.2 milliseconds, and it seems to be doing pretty well. The model identifies over 100 different kinds of aircraft (orange boxes) of many shapes and sizes, civilian and military, without missing a single one. It also identifies around 20 different types of helicopter (blue boxes) in the top right and even spots the cars on the highway and in the parking lots (red boxes). It’s not perfect– it thinks there’s a ship in the bottom left corner near the shed (yellow box); in reality this appears to be half of a plane’s fuselage, an understandable mistake given how long it took <em>me</em> to figure out what it was.</p>
|
||||
<p>This image shows a remarkable degree of accuracy being achieved by our model. Inference took just 822.2 milliseconds, and it seems to be doing pretty well. The model identifies over 100 different kinds of aircraft (orange boxes) of many shapes and sizes, civilian and military, without missing a single one. It also identifies around 20 different types of helicopter (blue boxes) in the top right and even spots the cars on the highway and in the parking lots (red boxes). It’s not perfect – it thinks there’s a ship in the bottom left corner near the shed (yellow box); in reality this appears to be half of a plane’s fuselage, an understandable mistake given how long it took <em>me</em> to figure out what it was.</p>
|
||||
<!--
|
||||
Even through we trained our model on Sentinel-2 imagery (10 meters per pixel), it can still be used on imagery from different satellites as long as they have a broadly similar resolution. A ship in PlanetScope imagery (3 meters per pixel) will look roughly similar to a ship in Sentinel-2 imagery. Using PlanetScope has another big over Sentinel-2 beyond its higher spatial resolution: it has a much higher revisit rate (daily instead of 5 days). Though *downloading* PlanetScope imagery isn't free, you *can* generate a timelapse image of any area on Earth using Planet's [Planet Stories](https://www.planet.com/stories/create) tool. Simply create a free account and follow the instructions to generate a timelapse of an area of interest. You can then download the timelapse video and use it as input to our model.
|
||||
Even though we trained our model on Sentinel-2 imagery (10 meters per pixel), it can still be used on imagery from different satellites as long as they have a broadly similar resolution. A ship in PlanetScope imagery (3 meters per pixel) will look roughly similar to a ship in Sentinel-2 imagery. Using PlanetScope has another big advantage over Sentinel-2 beyond its higher spatial resolution: it has a much higher revisit rate (daily instead of 5 days). Though *downloading* PlanetScope imagery isn't free, you *can* generate a timelapse image of any area on Earth using Planet's [Planet Stories](https://www.planet.com/stories/create) tool. Simply create a free account and follow the instructions to generate a timelapse of an area of interest. You can then download the timelapse video and use it as input to our model.
|
||||
|
||||
|
||||
Once you've done this, you can run the following line of code to automatically identify ships in the timelapse video:
|
||||
|
||||
@@ -561,7 +574,9 @@ Once you've done this, you can run the following line of code to automatically i
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
-->
|
||||
|
||||
@@ -820,8 +835,8 @@ window.document.addEventListener("DOMContentLoaded", function (event) {
|
||||
</script>
|
||||
<nav class="page-navigation">
|
||||
<div class="nav-page nav-page-previous">
|
||||
<a href="./blast.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text">Blast Damage Assessment</span>
|
||||
<a href="../chapters/C4_Ships.html" class="pagination-link">
|
||||
<i class="bi bi-arrow-left-short"></i> <span class="nav-page-text"><span class="chapter-title">Ship Detection</span></span>
|
||||
</a>
|
||||
</div>
|
||||
<div class="nav-page nav-page-next">
|
||||
|
Before Width: | Height: | Size: 1.0 MiB After Width: | Height: | Size: 1.0 MiB |
|
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 205 KiB After Width: | Height: | Size: 205 KiB |
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 56 KiB |
|
Before Width: | Height: | Size: 697 KiB After Width: | Height: | Size: 697 KiB |
|
Before Width: | Height: | Size: 809 KiB After Width: | Height: | Size: 809 KiB |
|
Before Width: | Height: | Size: 971 KiB After Width: | Height: | Size: 971 KiB |
|
Before Width: | Height: | Size: 635 KiB After Width: | Height: | Size: 635 KiB |
|
Before Width: | Height: | Size: 240 KiB After Width: | Height: | Size: 240 KiB |
|
Before Width: | Height: | Size: 945 KiB After Width: | Height: | Size: 945 KiB |
|
Before Width: | Height: | Size: 3.8 MiB After Width: | Height: | Size: 3.8 MiB |
|
Before Width: | Height: | Size: 3.2 MiB After Width: | Height: | Size: 3.2 MiB |
|
Before Width: | Height: | Size: 14 KiB After Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 29 KiB After Width: | Height: | Size: 29 KiB |
|
Before Width: | Height: | Size: 6.0 MiB After Width: | Height: | Size: 6.0 MiB |
|
Before Width: | Height: | Size: 31 KiB After Width: | Height: | Size: 31 KiB |
|
Before Width: | Height: | Size: 24 KiB After Width: | Height: | Size: 24 KiB |
|
Before Width: | Height: | Size: 96 KiB After Width: | Height: | Size: 96 KiB |
|
Before Width: | Height: | Size: 215 KiB After Width: | Height: | Size: 215 KiB |
|
Before Width: | Height: | Size: 1.9 MiB After Width: | Height: | Size: 1.9 MiB |
|
Before Width: | Height: | Size: 68 KiB After Width: | Height: | Size: 68 KiB |
|
Before Width: | Height: | Size: 1.1 MiB After Width: | Height: | Size: 1.1 MiB |
|
Before Width: | Height: | Size: 275 KiB After Width: | Height: | Size: 275 KiB |
|
Before Width: | Height: | Size: 2.2 MiB After Width: | Height: | Size: 2.2 MiB |
|
Before Width: | Height: | Size: 45 KiB After Width: | Height: | Size: 45 KiB |
|
Before Width: | Height: | Size: 911 KiB After Width: | Height: | Size: 911 KiB |
|
Before Width: | Height: | Size: 3.5 MiB After Width: | Height: | Size: 3.5 MiB |
|
Before Width: | Height: | Size: 3.8 MiB After Width: | Height: | Size: 3.8 MiB |
|
Before Width: | Height: | Size: 34 KiB After Width: | Height: | Size: 34 KiB |
|
Before Width: | Height: | Size: 482 KiB After Width: | Height: | Size: 482 KiB |
|
Before Width: | Height: | Size: 528 KiB After Width: | Height: | Size: 528 KiB |
|
Before Width: | Height: | Size: 232 KiB After Width: | Height: | Size: 232 KiB |
|
Before Width: | Height: | Size: 20 KiB After Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 2.3 MiB After Width: | Height: | Size: 2.3 MiB |
|
Before Width: | Height: | Size: 470 KiB After Width: | Height: | Size: 470 KiB |
|
Before Width: | Height: | Size: 79 KiB After Width: | Height: | Size: 79 KiB |
|
Before Width: | Height: | Size: 2.5 MiB After Width: | Height: | Size: 2.5 MiB |
|
Before Width: | Height: | Size: 2.3 MiB After Width: | Height: | Size: 2.3 MiB |
|
Before Width: | Height: | Size: 8.1 KiB After Width: | Height: | Size: 8.1 KiB |
|
Before Width: | Height: | Size: 770 KiB After Width: | Height: | Size: 770 KiB |
|
Before Width: | Height: | Size: 469 KiB After Width: | Height: | Size: 469 KiB |
|
Before Width: | Height: | Size: 77 KiB After Width: | Height: | Size: 77 KiB |
|
Before Width: | Height: | Size: 26 KiB After Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 744 KiB After Width: | Height: | Size: 744 KiB |
|
Before Width: | Height: | Size: 1.4 MiB After Width: | Height: | Size: 1.4 MiB |
|
Before Width: | Height: | Size: 74 KiB After Width: | Height: | Size: 74 KiB |
|
Before Width: | Height: | Size: 94 KiB After Width: | Height: | Size: 94 KiB |
|
Before Width: | Height: | Size: 149 KiB After Width: | Height: | Size: 149 KiB |
|
Before Width: | Height: | Size: 66 KiB After Width: | Height: | Size: 66 KiB |
|
Before Width: | Height: | Size: 687 KiB After Width: | Height: | Size: 687 KiB |
|
Before Width: | Height: | Size: 726 KiB After Width: | Height: | Size: 726 KiB |
|
Before Width: | Height: | Size: 173 KiB After Width: | Height: | Size: 173 KiB |
|
Before Width: | Height: | Size: 38 KiB After Width: | Height: | Size: 38 KiB |
|
Before Width: | Height: | Size: 6.7 MiB After Width: | Height: | Size: 6.7 MiB |
|
Before Width: | Height: | Size: 15 KiB After Width: | Height: | Size: 15 KiB |
|
Before Width: | Height: | Size: 2.5 KiB After Width: | Height: | Size: 2.5 KiB |
|
Before Width: | Height: | Size: 1.4 KiB After Width: | Height: | Size: 1.4 KiB |
|
Before Width: | Height: | Size: 1.2 KiB After Width: | Height: | Size: 1.2 KiB |
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user