% Options for packages loaded elsewhere
\PassOptionsToPackage{unicode}{hyperref}
\PassOptionsToPackage{hyphens}{url}
\PassOptionsToPackage{dvipsnames,svgnames,x11names}{xcolor}
%
\documentclass[
  letterpaper,
  DIV=11,
  numbers=noendperiod,
  oneside]{scrreprt}

\usepackage{amsmath,amssymb}
\usepackage{lmodern}
\usepackage{iftex}
\ifPDFTeX
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{textcomp} % provide euro and other symbols
\else % if luatex or xetex
  \usepackage{unicode-math}
  \defaultfontfeatures{Scale=MatchLowercase}
  \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}
\fi
% Use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\IfFileExists{microtype.sty}{% use microtype if available
  \usepackage[]{microtype}
  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\makeatletter
\@ifundefined{KOMAClassName}{% if non-KOMA class
  \IfFileExists{parskip.sty}{%
    \usepackage{parskip}
  }{% else
    \setlength{\parindent}{0pt}
    \setlength{\parskip}{6pt plus 2pt minus 1pt}}
}{% if KOMA class
  \KOMAoptions{parskip=half}}
\makeatother
\usepackage{xcolor}
\usepackage[left=1in,marginparwidth=2.0666666666667in,textwidth=4.1333333333333in,marginparsep=0.3in]{geometry}
\setlength{\emergencystretch}{3em} % prevent overfull lines
\setcounter{secnumdepth}{5}
% Make \paragraph and \subparagraph free-standing
\ifx\paragraph\undefined\else
  \let\oldparagraph\paragraph
  \renewcommand{\paragraph}[1]{\oldparagraph{#1}\mbox{}}
\fi
\ifx\subparagraph\undefined\else
  \let\oldsubparagraph\subparagraph
  \renewcommand{\subparagraph}[1]{\oldsubparagraph{#1}\mbox{}}
\fi

\usepackage{color}
\usepackage{fancyvrb}
\newcommand{\VerbBar}{|}
\newcommand{\VERB}{\Verb[commandchars=\\\{\}]}
\DefineVerbatimEnvironment{Highlighting}{Verbatim}{commandchars=\\\{\}}
% Add ',fontsize=\small' for more characters per line
\usepackage{framed}
\definecolor{shadecolor}{RGB}{241,243,245}
\newenvironment{Shaded}{\begin{snugshade}}{\end{snugshade}}
\newcommand{\AlertTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\AnnotationTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{#1}}
\newcommand{\AttributeTok}[1]{\textcolor[rgb]{0.40,0.45,0.13}{#1}}
\newcommand{\BaseNTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\BuiltInTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\CharTok}[1]{\textcolor[rgb]{0.13,0.47,0.30}{#1}}
\newcommand{\CommentTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{#1}}
\newcommand{\CommentVarTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{\textit{#1}}}
\newcommand{\ConstantTok}[1]{\textcolor[rgb]{0.56,0.35,0.01}{#1}}
\newcommand{\ControlFlowTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\DataTypeTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\DecValTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\DocumentationTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{\textit{#1}}}
\newcommand{\ErrorTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\ExtensionTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\FloatTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\FunctionTok}[1]{\textcolor[rgb]{0.28,0.35,0.67}{#1}}
\newcommand{\ImportTok}[1]{\textcolor[rgb]{0.00,0.46,0.62}{#1}}
\newcommand{\InformationTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{#1}}
\newcommand{\KeywordTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\NormalTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\OperatorTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{#1}}
\newcommand{\OtherTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\PreprocessorTok}[1]{\textcolor[rgb]{0.68,0.00,0.00}{#1}}
\newcommand{\RegionMarkerTok}[1]{\textcolor[rgb]{0.00,0.23,0.31}{#1}}
\newcommand{\SpecialCharTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{#1}}
\newcommand{\SpecialStringTok}[1]{\textcolor[rgb]{0.13,0.47,0.30}{#1}}
\newcommand{\StringTok}[1]{\textcolor[rgb]{0.13,0.47,0.30}{#1}}
\newcommand{\VariableTok}[1]{\textcolor[rgb]{0.07,0.07,0.07}{#1}}
\newcommand{\VerbatimStringTok}[1]{\textcolor[rgb]{0.13,0.47,0.30}{#1}}
\newcommand{\WarningTok}[1]{\textcolor[rgb]{0.37,0.37,0.37}{\textit{#1}}}

\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}\usepackage{longtable,booktabs,array}
\usepackage{calc} % for calculating minipage widths
% Correct order of tables after \paragraph or \subparagraph
\usepackage{etoolbox}
\makeatletter
\patchcmd\longtable{\par}{\if@noskipsec\mbox{}\fi\par}{}{}
\makeatother
% Allow footnotes in longtable head/foot
\IfFileExists{footnotehyper.sty}{\usepackage{footnotehyper}}{\usepackage{footnote}}
\makesavenoteenv{longtable}
\usepackage{graphicx}
\makeatletter
\def\maxwidth{\ifdim\Gin@nat@width>\linewidth\linewidth\else\Gin@nat@width\fi}
\def\maxheight{\ifdim\Gin@nat@height>\textheight\textheight\else\Gin@nat@height\fi}
\makeatother
% Scale images if necessary, so that they will not overflow the page
% margins by default, and it is still possible to overwrite the defaults
% using explicit options in \includegraphics[width, height, ...]{}
\setkeys{Gin}{width=\maxwidth,height=\maxheight,keepaspectratio}
% Set default figure placement to htbp
\makeatletter
\def\fps@figure{htbp}
\makeatother

\KOMAoption{captions}{tableheading}
\makeatletter
\@ifpackageloaded{tcolorbox}{}{\usepackage[many]{tcolorbox}}
\@ifpackageloaded{fontawesome5}{}{\usepackage{fontawesome5}}
\definecolor{quarto-callout-color}{HTML}{909090}
\definecolor{quarto-callout-note-color}{HTML}{0758E5}
\definecolor{quarto-callout-important-color}{HTML}{CC1914}
\definecolor{quarto-callout-warning-color}{HTML}{EB9113}
\definecolor{quarto-callout-tip-color}{HTML}{00A047}
\definecolor{quarto-callout-caution-color}{HTML}{FC5300}
\definecolor{quarto-callout-color-frame}{HTML}{acacac}
\definecolor{quarto-callout-note-color-frame}{HTML}{4582ec}
\definecolor{quarto-callout-important-color-frame}{HTML}{d9534f}
\definecolor{quarto-callout-warning-color-frame}{HTML}{f0ad4e}
\definecolor{quarto-callout-tip-color-frame}{HTML}{02b875}
\definecolor{quarto-callout-caution-color-frame}{HTML}{fd7e14}
\makeatother
\makeatletter
\makeatother
\makeatletter
\@ifpackageloaded{bookmark}{}{\usepackage{bookmark}}
\makeatother
\makeatletter
\@ifpackageloaded{caption}{}{\usepackage{caption}}
\AtBeginDocument{%
\ifdefined\contentsname
  \renewcommand*\contentsname{Table of contents}
\else
  \newcommand\contentsname{Table of contents}
\fi
\ifdefined\listfigurename
  \renewcommand*\listfigurename{List of Figures}
\else
  \newcommand\listfigurename{List of Figures}
\fi
\ifdefined\listtablename
  \renewcommand*\listtablename{List of Tables}
\else
  \newcommand\listtablename{List of Tables}
\fi
\ifdefined\figurename
  \renewcommand*\figurename{Figure}
\else
  \newcommand\figurename{Figure}
\fi
\ifdefined\tablename
  \renewcommand*\tablename{Table}
\else
  \newcommand\tablename{Table}
\fi
}
\@ifpackageloaded{float}{}{\usepackage{float}}
\floatstyle{ruled}
\@ifundefined{c@chapter}{\newfloat{codelisting}{h}{lop}}{\newfloat{codelisting}{h}{lop}[chapter]}
\floatname{codelisting}{Listing}
\newcommand*\listoflistings{\listof{codelisting}{List of Listings}}
\makeatother
\makeatletter
\@ifpackageloaded{caption}{}{\usepackage{caption}}
\@ifpackageloaded{subcaption}{}{\usepackage{subcaption}}
\makeatother
\makeatletter
\@ifpackageloaded{tcolorbox}{}{\usepackage[many]{tcolorbox}}
\makeatother
\makeatletter
\@ifundefined{shadecolor}{\definecolor{shadecolor}{rgb}{.97, .97, .97}}
\makeatother
\makeatletter
\@ifpackageloaded{sidenotes}{}{\usepackage{sidenotes}}
\@ifpackageloaded{marginnote}{}{\usepackage{marginnote}}
\makeatother
\makeatletter
\makeatother
\ifLuaTeX
  \usepackage{selnolig}  % disable illegal ligatures
\fi
\IfFileExists{bookmark.sty}{\usepackage{bookmark}}{\usepackage{hyperref}}
\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available
\urlstyle{same} % disable monospaced font for URLs
\hypersetup{
  pdftitle={Google Earth Engine for OSINT},
  pdfauthor={Dr.~Ollie Ballinger},
  colorlinks=true,
  linkcolor={blue},
  filecolor={Maroon},
  citecolor={Blue},
  urlcolor={Blue},
  pdfcreator={LaTeX via pandoc}}

\title{Google Earth Engine for OSINT}
\author{\href{https://oballinger.github.io}{Dr.~Ollie Ballinger}}
\date{10/6/22}

\begin{document}
\maketitle
\ifdefined\Shaded\renewenvironment{Shaded}{\begin{tcolorbox}[breakable, sharp corners, boxrule=0pt, borderline west={3pt}{0pt}{shadecolor}, enhanced, frame hidden, interior hidden]}{\end{tcolorbox}}\fi

\renewcommand*\contentsname{Table of contents}
{
\hypersetup{linkcolor=}
\setcounter{tocdepth}{2}
\tableofcontents
}
\bookmarksetup{startatroot}

\hypertarget{introduction}{%
\chapter*{Introduction}\label{introduction}}
\addcontentsline{toc}{chapter}{Introduction}

\markboth{Introduction}{Introduction}

The analysis of satellite imagery is a foundational element of open
source investigations. In the past decade, the quantity, quality, and
availability thereof has increased dramatically. Capabilities and
insights that were once only available to governments are now accessible
to the general public. Satellite imagery is being used to collect
evidence of genocide and other war crimes in
\href{https://www.nbcnews.com/science/science-news/ukraine-satellites-war-crimes-rcna26291}{Ukraine},
\href{https://www.amnesty.org/en/latest/news/2016/04/nigeria-military-cover-up-of-mass-slaughter-at-zaria-exposed/}{Nigeria},
\href{https://www.amnesty.org/en/latest/news/2016/01/burundi-satellite-evidence-supports-witness-accounts-of-mass-graves/}{Burundi},
\href{https://www.amnesty.org/en/latest/news/2021/07/cameroon-satellite-images-reveal-devastation-in-anglophone-regions-2/}{Cameroon},
\href{https://www.aaas.org/resources/satellite-imagery-assessment-forced-relocations-near-luiswishi-mine}{the
DRC},
\href{https://gsp.yale.edu/case-studies/sudan/maps-satellite-images/other-darfur-satellite-imagery}{South
Sudan},
\href{https://gsp.yale.edu/resources/maps-satellite-images/papua}{Papua},
and
\href{https://www.hrw.org/report/2016/04/04/unchecked-power/police-and-military-raids-low-income-and-immigrant-communities}{Venezuela}.
It has been used to
\href{https://www.theguardian.com/environment/2016/mar/02/new-satellite-mapping-a-game-changer-against-illegal-logging}{monitor
environmental degradation} and hold extractive industries to account
from
\href{https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/}{Iraq}
to
\href{https://www.planet.com/pulse/the-observatory-of-extractive-industries-oie-shines-a-light-on-the-mining-industry-using-planets-satellite-data/}{Guatemala}.
The ability to analyze satellite imagery is a critical skill for anyone
interested in open source investigations.

Though no-code platforms such as Sentinelhub have been invaluable in
allowing the OSINT community to access and process satellite imagery,
the analytical capabilities of these platforms are limited.
\href{https://earthengine.google.com/\#intro}{Google Earth Engine (GEE)}
is a cloud-based platform that stores petabytes of satellite imagery
from a variety of sources and allows users to perform advanced analyses
on Google servers for free using a browser-based interface. This
textbook is designed for investigators who want to perform more
sophisticated analysis using geospatial data, and assumes no prior
knowledge of coding or remote sensing (satellite imagery analysis). It
is organized into two parts: an introduction to remote sensing and GEE,
and a series of case studies that demonstrate how to use GEE for open
source investigations.

\hypertarget{table-of-contents}{%
\section*{Table of Contents}\label{table-of-contents}}
\addcontentsline{toc}{section}{Table of Contents}

\markright{Table of Contents}

\begin{itemize}
\tightlist
\item
  Learning

  \begin{enumerate}
  \def\labelenumi{\arabic{enumi}.}
  \tightlist
  \item
    \protect\hyperlink{remote-sensing}{Remote Sensing}
  \item
    \protect\hyperlink{data-acquisition}{Data Acquisition}
  \item
    \href{ch4.qmd}{Application Development}
  \end{enumerate}
\item
  Case Studies

  \begin{enumerate}
  \def\labelenumi{\arabic{enumi}.}
  \tightlist
  \item
    \protect\hyperlink{war-at-night}{War at Night}
  \item
    \protect\hyperlink{refinery-identification}{Refinery Detection}
  \item
    \href{trees.qmd}{Deforestation}
  \item
    \protect\hyperlink{ship-detection}{Ship Detection}
  \item
    \href{object_detection.qmd}{Object Detection} Recently, a team of
    over 100 scientists came together to write a book called
    \href{https://www.eefabook.org/}{``Cloud-Based Remote Sensing with
    Google Earth Engine: Fundamentals and Applications''}. It's a great
    resource for learning about remote sensing and Earth Engine. The
    material in this chapter is a subset of the book, edited to fit the
    scope of this guide. If you're interested in learning more, check
    out the full book.
  \end{enumerate}
\end{itemize}

\hypertarget{what-is-google-earth-engine}{%
\section*{What is Google Earth
Engine?}\label{what-is-google-earth-engine}}
\addcontentsline{toc}{section}{What is Google Earth Engine?}

\markright{What is Google Earth Engine?}

As geospatial datasets---particularly satellite imagery
collections---increase in size, researchers are increasingly relying on
cloud computing platforms such as Google Earth Engine (GEE) to analyze
vast quantities of data.

GEE is free and allows users to write open-source code that can be run
by others in one click, thereby yielding fully reproducible results.
These features have put GEE on the cutting edge of scientific research.
The following plot visualizes the number of journal articles conducted
using different geospatial analysis software platforms:

\includegraphics{././images/WoS Articles.png}

Despite only being released in 2015, the number of geospatial journal
articles using Google Earth Engine (shown in red above) has outpaced
every other major geospatial analysis software, including ArcGIS,
Python, and R in just five years. GEE applications have been developed
and used to present interactive geospatial data visualizations by NGOs,
Universities, the United Nations, and the European Commission. By
storing and running computations on google servers, GEE is far more
accessible to those who don't have significant local computational
resources; all you need is an internet connection.

\part{Learning}

\hypertarget{remote-sensing}{%
\chapter{Remote Sensing}\label{remote-sensing}}

Before learning how to load, process, and analyze satellite imagery in
Google Earth Engine, it will be helpful to know a few basic principles
of remote sensing. This section provides a brief overview of some
important concepts and terminology that will be used throughout the
course, including active and passive sensors; spatial, spectral, and
temporal resolution; and orbits.

\hypertarget{active-and-passive-sensors}{%
\section{Active and Passive Sensors}\label{active-and-passive-sensors}}

\href{https://www.sciencedirect.com/topics/medicine-and-dentistry/remote-sensing}{Remote
sensing} is the science of obtaining information about an object or
phenomenon without making physical contact with the object. Remote
sensing can be done with various types of electromagnetic radiation such
as visible, infrared, or microwave. The electromagnetic radiation is
either emitted or reflected from the object being sensed. The reflected
radiation is then collected by a sensor and processed to obtain
information about the object.

\includegraphics{././images/diagram.png}

While most satellite imagery is optical, meaning it captures sunlight
reflected by the earth's surface, Synthetic Aperture Radar (SAR)
satellites such as Sentinel-1 work by emitting pulses of radio waves and
measuring how much of the signal is reflected back. This is similar to
the way a bat uses sonar to ``see'' in the dark: by emitting calls and
listening to echoes.

\hypertarget{resolution}{%
\section{Resolution}\label{resolution}}

Resolution is one of the most important attributes of satellite imagery.

here are three types of resolution: spatial, spectral, and temporal.

\hypertarget{spatial-resolution}{%
\subsection{Spatial Resolution}\label{spatial-resolution}}

Spatial resolution governs how ``sharp'' an image looks. The Google Maps
satellite basemap, for example, is really sharp Most of the optical
imagery that is freely available has relatively low spatial resolution
(it looks more grainy than, for example, the Google satellite basemap),

\includegraphics{././images/Landsat.png}
\includegraphics{././images/Sentinel2.png}
\includegraphics{././images/Maxar.png}

\hypertarget{spectral-resolution}{%
\subsection{Spectral Resolution}\label{spectral-resolution}}

What open source imagery lacks in spatial resolution it often makes up
for with \emph{spectral} resolution. Really sharp imagery from MAXAR,
for example, collects

Different materials reflect light differently. An apple absorbs shorter
wavelengths (e.g.~blue and green), and reflects longer wavelengths
(red). Our eyes use that information-- the color-- to distinguish
between different objects. But our eyes can only see a relatively small
sliver of the electromagnetic spectrum covering blue, yellow, and red;
we can't see UV or infrared wavelengths, for example, though the extent
to which different materials reflect or absorb these wavelengths is just
as useful for distinguishing between them. For example, Astroturf (fake
plastic grass) and real grass will both look green to us, espeically
from a satellite image. But living plants absorb radiation from the sun
in a part of the light spectrum that we can't see. There's a spectral
index called the Normalized Difference Vegetation Index (NDVI) which
exploits this fact to isolate vegetation in multispectral satellite
imagery. So if we look at
\href{https://en.wikipedia.org/wiki/Gillette_Stadium}{Gilette Stadium}
near Boston, we can tell that the three training fields south of the
stadium are real grass (they generate high NDVI values, showing up red),
while the pitch in the stadium itself is astroturf (generating low NDVI
values, showing up blue).

\begin{figure}

{\centering \includegraphics{./images/NDVI.jpg}

}

\caption{VHR image of Gilette Stadium with Sentinel-2 derived NDVI
overlay}

\end{figure}

In other words, even though these fields are all green and
indistinguishable to the human eye, their \emph{spectral profiles}
beyond the visible light spectrum differ, and we can use this
information to distinguish between them. Below is a plot of the spectral
profiles of different materials, including oil.

The European Space Agency's Sentinel-2 satellite collects spectral
information well beyond the visible light spectrum, enabling this sort
of analysis. It chops the electromagnetic spectrum up into ``bands'',
and measures how strongly wavelengths in each of those bands is
reflected:

\includegraphics{./images/S2_bands.png}

We'll be using this satellite to distinguish between oil and other
materials, similar to the way we were able to distinguish between real
and fake grass at Gilette Stadium. First, we'll have to do a bit of
pre-processing on the Sentinel-2 imagery after which we'll train a
machine learning model to identify oil.

\hypertarget{temporal-resolution}{%
\subsection{Temporal Resolution}\label{temporal-resolution}}

Finally, the frequency with which we There is often a tradeoff between
spatial and temporal resolution.

The Google Maps basemap is very high resolution, available globally, and
is freely available. But it has no \emph{temporal} dimension: it's a
snapshot from one particular point in time. If the thing we're
interested in involves \emph{changes} over time, this basemap will be of
limited use.

The \textbf{``revisit rate''} is the amount of time it takes for the
satellite to pass over the same location twice. The revisit rate is
inversely proportional to the satellite's altitude: the higher the
satellite is, the more frequently it can pass over the same location.
This generally means that there's a tradeoff between spatial resolution
and temporal resolution: the higher the spatial resolution, the lower
the revisit rate. However, some satellite constellations such as
Planet's SkySat are able to achieve both high spatial and temporal
resolution by launching lots of small satellites into orbit at once.
Below is a comparison of revisit rates for various satellites:

\begin{itemize}
\tightlist
\item
  \href{https://sentinels.copernicus.eu/web/sentinel/user-guides/sentinel-1-sar/revisit-and-coverage}{Sentinel
  1}: 3 days (6 days as of 23/12/21, since Sentinel-1B was decomisioned)
\item
  \href{https://sentinel.esa.int/web/sentinel/missions/sentinel-2}{Sentinel
  2}: 5 days
\item
  \href{https://landsat.gsfc.nasa.gov/satellites/landsat-9/\#:~:text=Landsat\%209\%20replaces\%20Landsat\%207,for\%20Landsat\%208\%20\%2B\%20Landsat\%207.}{Landsat
  8-9}: 8 days
\item
  \href{https://www.planet.com/pulse/12x-rapid-revisit-announcement/}{Planet
  SkySat}: 2-3 hours
\end{itemize}

\hypertarget{orbits}{%
\section{Orbits}\label{orbits}}

The Landsat satellites are in a sun-synchronous orbit, meaning they pass
over the same spot on Earth at the same time every day. The Sentinel
satellites are in a polar orbit, meaning they pass over the same spot on
Earth twice a day, once in the morning and once in the afternoon. NASA
have created a great
\href{https://svs.gsfc.nasa.gov/4745}{visualisation} showing the orbits
of the Landsat and Sentinel-2 satellites:

\url{https://svs.gsfc.nasa.gov/vis/a000000/a004700/a004745/landsat_w_sentinel_ls8ls9sAsB_fade_1080p60.mp4}

The Sentinel satellites are in a lower orbit than Landsat, meaning they
are closer to the Earth and have a higher resolution.

\hypertarget{data-acquisition}{%
\chapter{Data Acquisition}\label{data-acquisition}}

One of the main advantages of GEE is that it hosts several Petabytes of
satellite imagery and other spatial data sets,
\href{https://developers.google.com/earth-engine/datasets}{all in one
place}. Among these are a many that could prove useful to those
investigating illegal mining and logging, estimating conflict-induced
damage, monitoring pollution from extractive industries, conducting
maritime surveillance without relying on ship transponders, verifying
the locations of artillery strikes, tracking missile defense systems,
and many other topics.

This section highlights ten categories of geospatial data available
natively in the GEE catalogue ranging from optical satellite imagery, to
atmospheric data, to building footprints. Each sub-section provides an
overview of the given data type, suggests potential applications, and
lists the corresponding datasets in the GEE catalogue. The datasets
listed under each heading are \textbf{not} an exhaustive list-- there
are over 500 in the whole catalogue, and the ones listed in this section
are simply the ones with the most immediate relevance to open source
investigations. If a particular geospatial dataset you want to work with
isn't hosted in the GEE catalog, you can upload your own data. We'll
cover that in the next section.

\begin{figure}

{\centering \includegraphics{././images/hasankeyf.gif}

}

\caption{Sentinel-2 timelapse showing the ancient city of Hasankeyf
being flooded following the construction of a dam by the Turkish
government.}

\end{figure}

Optical satellite imagery is the bread and butter of many open source
investiagtions. It would be tough to list off all of the possible use
cases, so here's a handy flowchart:

\begin{figure}[H]

{\centering \includegraphics[width=5.69in,height=3.99in]{./ch2_files/figure-latex/mermaid-figure-1.png}

}

\end{figure}

This is, of course, a bit of an exaggeration. But if you're interested
in a visible phenomenon that happens outdoors and that isn't very tiny,
chances are an earth-observing satellite has taken a picture of it. What
that picture can tell you naturally depends on what you're interested in
learning. For a deeper dive into analyzing optical satellite imagery,
see the subsection on
\protect\hyperlink{multispectral-remote-sensing-remote_sensing}{multispectral
remote sensing.}.

There are several different types of optical satellite imagery available
in the GEE catalogue. The main collections are the Landsat and Sentinel
series of satellites, which are operated by NASA and the European Space
Agency, respectively. Landsat satellites have been in orbit since 1972,
and Sentinel satellites have been in orbit since 2015. Norway's
International Climate and Forest Initiative (NICFI) has also contributed
to the GEE catalogue by providing a collection of optical imagery from
Planet's PlanetScope satellites. These are higher resolution (4.7 meters
per pixel) than Landsat (30m/px) and Sentinel-2 (10m/px), but are only
available for the tropics. Even higher resolution imagery (60cm/px) is
available from the GEE catalogue from the National Agriculture Imagery
Program, but it is only available for the United States. For more
details, see the ``Datasets'' section below.

\hypertarget{applications}{%
\subsection*{Applications}\label{applications}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Geolocating pictures

  \begin{itemize}
  \tightlist
  \item
    Some of Bellingcat's
    \href{https://www.bellingcat.com/resources/how-tos/2014/07/09/verification-and-geolocation-tricks-and-tips-with-google-earth/}{earliest
    work} involved figuring out where a picture was taken by
    cross-referencing it with optical satellite imagery.
  \end{itemize}
\item
  General surveillance

  \begin{itemize}
  \tightlist
  \item
    \href{https://web.archive.org/web/20220415054905/https://fas.org/blogs/security/2021/11/a-closer-look-at-chinas-missile-silo-construction/}{Monitoring}
    Chinese missile silo construction.
  \item
    Amassing
    \href{https://www.nytimes.com/2022/04/04/world/europe/bucha-ukraine-bodies.html}{evidence}
    of genocide in Bucha, Ukraine
  \end{itemize}
\item
  Damage detection

  \begin{itemize}
  \tightlist
  \item
    \href{https://www.theguardian.com/world/2022/oct/27/before-and-after-satellite-imagery-will-track-ukraine-cultural-damage-un-says}{Ukraine}
  \item
    \href{https://reliefweb.int/report/mali/satellite-imagery-conflict-affected-areas-how-technology-can-support-wfp-emergency}{Mali}
  \item
    \href{https://www.pnas.org/doi/pdf/10.1073/pnas.2025400118}{Around
    the World}
  \end{itemize}
\item
  Verifying the locations of artillery/missile/drone strikes

  \begin{itemize}
  \tightlist
  \item
    The
    \href{https://www.cnbc.com/2019/09/17/satellite-photos-show-extent-of-damage-to-saudi-aramco-plants.html}{2019
    attack} on Saudi Arabia's Abqaiq oil processing facility.
  \end{itemize}
\item
  Monitoring illegal mining/logging

  \begin{itemize}
  \tightlist
  \item
    Global Witness
    \href{https://www.globalwitness.org/en/campaigns/natural-resource-governance/myanmars-poisoned-mountains/}{investigation}
    into illegal mining by militias in Myanmar.
  \item
    Tracking
    \href{https://www.theguardian.com/environment/2016/mar/02/new-satellite-mapping-a-game-changer-against-illegal-logging}{illegal
    logging} across the world.
  \end{itemize}
\end{itemize}

\hypertarget{datasets}{%
\subsection*{Datasets}\label{datasets}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/landsat-mss}{Landsat
1-5} & 1972--1999 & 30m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LE07_C02_T1_L2}{Landsat
7} & 1999--2021 & 30m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC08_C02_T1_L2}{Landsat
8} & 2013--Present & 30m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC09_C02_T1_L2}{Landsat
9} & 2021--Present & 30m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR_HARMONIZED}{Sentinel-2}
& 2015--Present & 10m & Global \\
\href{https://developers.google.com/earth-engine/datasets/tags/nicfi}{NICFI}
& 2015-Present & 4.7m & Tropics \\
\href{https://developers.google.com/earth-engine/datasets/catalog/USDA_NAIP_DOQQ}{NAIP}
& 2002-2021 & 0.6m & USA \\
\bottomrule()
\end{longtable}

\hypertarget{radar-imagery}{%
\section{Radar Imagery}\label{radar-imagery}}

\begin{figure}

{\centering \includegraphics{././images/radar ships.jpg}

}

\caption{Ships and interference from a radar system are visible in
Zhuanghe Wan, near North Korea.}

\end{figure}

Synthetic Aperture Radar imagery (SAR) is a type of remote sensing that
uses radio waves to detect objects on the ground. SAR imagery is useful
for detecting objects that are small, or that are obscured by clouds or
other weather phenomena. SAR imagery is also useful for detecting
objects that are moving, such as ships or cars.

\hypertarget{applications-1}{%
\subsection*{Applications}\label{applications-1}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Change/Damage detection
\item
  Tracking military radar systems
\item
  Maritime surveillance
\item
  Monitoring illegal mining/logging
\end{itemize}

\hypertarget{datasets-1}{%
\subsection*{Datasets}\label{datasets-1}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S1_GRD}{Sentinel
1} & 2014-Present & 10m & Global \\
\bottomrule()
\end{longtable}

\hypertarget{nighttime-lights}{%
\section{Nighttime Lights}\label{nighttime-lights}}

\begin{figure}

{\centering \includegraphics{././images/Figure_1.gif}

}

\caption{A timelapse of nighttime lights over Northern Iraq showing the
capture and liberation of Mosul by ISIS.}

\end{figure}

Satellite images of the Earth at night a useful proxy for human
activity. The brightness of a given area at night is a function of the
number of people living there and the nature of their activities. The
effects of conflict, natural disasters, and economic development can all
be inferred from changes in nighttime lights.

The timelapse above reveals a number of interesting things: The capture
of Mosul by ISIS in 2014 and the destruction of its infrastructure
during the fighting (shown as the city darkening), as well as the
liberation of the city by the Iraqi military in 2017 are all visible in
nighttime lights. The code to create this gif, as well as a more
in-depth tutorial on the uses of nighttime lights, can be found in the
\protect\hyperlink{war-at-night}{``War at Night''} case study.

\hypertarget{applications-2}{%
\subsection*{Applications}\label{applications-2}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Damage detection
\item
  Identifying gas flaring/oil production
\item
  Identifying urban areas/military bases illuminated at night
\end{itemize}

\hypertarget{datasets-2}{%
\subsection*{Datasets}\label{datasets-2}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/NOAA_DMSP-OLS_NIGHTTIME_LIGHTS}{DMSP-OLS}
& 1992-2014 & 927m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/NOAA_VIIRS_DNB_MONTHLY_V1_VCMSLCFG}{VIIRS}
& 2014-Present & 463m & Global \\
\bottomrule()
\end{longtable}

\hypertarget{climate-and-atmospheric-data}{%
\section{Climate and Atmospheric
Data}\label{climate-and-atmospheric-data}}

\begin{figure}

{\centering \includegraphics{././images/mishraq_small.gif}

}

\caption{Sulphur Dioxide plume resulting from ISIS attack on the
Al-Mishraq Sulphur Plant in Iraq}

\end{figure}

Climate and atmospheric data can be used to track the effects of
conflict on the environment. The European Space Agency's Sentinel-5p
satellites measure the concentration of a number of atmospheric gases,
including nitrogen dioxide, methane, and ozone. Measurements are
available on a daily basis at a fairly high resolution (1km), allowing
for the detection of localized sources of pollution such as oil
refineries or power plants. For example, see this
\href{https://www.bellingcat.com/resources/2021/04/15/what-oil-satellite-technology-and-iraq-can-tell-us-about-pollution/}{Bellingcat
article} in which Wim Zwijnenburg and I trace pollution to specific
facilities operated by multinational oil companies in Iraq.

The Copernicus Atmosphere Monitoring Service (CAMS) provides similar
data at a lower spatial resolution (45km), but measurements are
avaialble on an hourly basis. The timelapse above utilizes CAMS data to
show a sulphur dioxide plume resulting from an ISIS attack on the
Al-Mishraq Sulphur Plant in Iraq. The plant was used to produce
sulphuric acid, for use in fertilizers and pesticides. The attack
destroyed the plant, causing a fire which burned for a month and
released
\href{https://earthobservatory.nasa.gov/images/88994/sulfur-dioxide-spreads-over-iraq}{21
kilotons} of sulphur dioxide into the atmosphere per day; the largest
human-made release of sulphur dioxide in history.

\hypertarget{applications-3}{%
\subsection*{Applications}\label{applications-3}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Monitoring of airborne pollution
\item
  Tracing pollution back to specific facilities and companies
\item
  Visualizing the effects of one-off environmental catastrophes

  \begin{itemize}
  \tightlist
  \item
    Nordstream 1 leak
  \item
    ISIS setting Mishraq sulphur plant on fire
  \end{itemize}
\end{itemize}

\hypertarget{datasets-3}{%
\subsection*{Datasets}\label{datasets-3}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/ECMWF_CAMS_NRT}{CAMS
NRT} & 2016-Present & 44528m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/sentinel-5p}{Sentinel-5p}
& 2018-Present & 1113m & Global \\
\bottomrule()
\end{longtable}

\hypertarget{mineral-deposits}{%
\section{Mineral Deposits}\label{mineral-deposits}}

\begin{figure}

{\centering \includegraphics{././images/mining.jpg}

}

\caption{Zinc deposits across Central Africa}

\end{figure}

Mining activities often play an important role in conflict. According to
an influential
\href{https://www.aeaweb.org/articles?id=10.1257/aer.20150774}{study},
``the historical rise in mineral prices might explain up to one-fourth
of the average level of violence across African countries'' between 1997
and 2010. Data on the location of mineral deposits can be used to
identify areas where mining activities are likely to be taking place,
and several such datasets are available in Google Earth Engine.

\hypertarget{applications-4}{%
\subsection*{Applications}\label{applications-4}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Monitoring mining activity
\item
  Identifying areas where mining activities are likely to be taking
  place
\item
  Mapping the distribution of resources in rebel held areas in conflicts
  fueled by resource extraction
\end{itemize}

\hypertarget{datasets-4}{%
\subsection*{Datasets}\label{datasets-4}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/tags/isda}{iSDA}
& 2001-2017 & 30m & Africa \\
\bottomrule()
\end{longtable}

\hypertarget{fires}{%
\section{Fires}\label{fires}}

\begin{figure}

{\centering \includegraphics{././images/fires.jpg}

}

\caption{Detected fires over Ukraine since 27/02/2022 showing the
frontline of the war}

\end{figure}

Earth-observing satellites can detect ``thermal anomalies'' (fires) from
space. NASA's Fire Information for Resource Management System (FIRMS)
provides daily data on active fires in near real time, going back to the
year 2000. Carlos Gonzales wrote a comprehensive
\href{https://www.bellingcat.com/resources/2022/10/04/scorched-earth-using-nasa-fire-data-to-monitor-war-zones/}{Bellingcat
article} on the use of FIRMS to monitor war zones from Ukraine to
Ethiopia. The map above shows that FIRMS detected fires over Eastern
Ukraine trace the frontline of the war.

FIRMS data are derived from the MODIS satellite, but only show the
central location and intensity of a detected fire. Another MODIS product
(linked in the table below) generates a monthly map of burned areas,
which can be used to assess the spatial extent of fires.

\hypertarget{applications-5}{%
\subsection*{Applications}\label{applications-5}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Identification of possible artillery strikes/fighting in places like
  Ukraine
\item
  Environmental warfare and ``scorched earth'' policies
\item
  Large scale arson

  \begin{itemize}
  \tightlist
  \item
    e.g.~\href{https://citizenevidence.org/2021/02/26/using-viirs-fire-data-for-human-rights-research/}{Refugee
    camps burned down in Myanmar}
  \end{itemize}
\end{itemize}

\hypertarget{datasets-5}{%
\subsection*{Datasets}\label{datasets-5}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/FIRMS}{FIRMS}
& 2000-Present & 1000m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/CIESIN_GPWv411_GPW_Population_Count}{MODIS
Burned Area} & 2000-Present & 500m & Global \\
\bottomrule()
\end{longtable}

\hypertarget{population-density-estimates}{%
\section{Population Density
Estimates}\label{population-density-estimates}}

\begin{figure}

{\centering \includegraphics{././images/pop.jpg}

}

\caption{Population density estimates around Pyongyang, North Korea}

\end{figure}

Sometimes, we may want to get an estimate the population in a specific
area to ballpark how many people might be affected by a natural
disaster, a counteroffensive, or a missile strike. You can't really
google ``what is the population in this rectangle i've drawn in
Northeastern Syria?'' and get a good answer. Luckily, there are several
spatial population datasets hosted in GEE that let you do just that.
Some, such as WorldPop, provide estimated breakdowns by age and sex as
well. However, it is extremely important to bear in mind that these are
\textbf{estimates}, and will \textbf{not} take into account things like
conflict-induced displacement. For example, Oak Ridge National
Laboratory's LandScan program has released high-resolution population
data for Ukraine, but this pertains to the pre-war population
distribution. The war has radically changed this distribution, so these
estimates no longer reflect where people \emph{are}. Still, this dataset
could be used to roughly estimate displacement or the number of people
who will need new housing.

\hypertarget{applications-6}{%
\subsection*{Applications:}\label{applications-6}}
\addcontentsline{toc}{subsection}{Applications:}

\begin{itemize}
\tightlist
\item
  Rough estimates of civilians at risk from conflict or disaster,
  provided at a high spatial resolution
\end{itemize}

\hypertarget{datasets-6}{%
\subsection*{Datasets}\label{datasets-6}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2683}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2927}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.2439}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 6\tabcolsep) * \real{0.1951}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Sensor
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Resolution
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/tags/worldpop}{Worldpop}
& 2000-2021 & 92m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/CIESIN_GPWv411_GPW_Population_Count}{GPW}
& 2000-2021 & 927m & Global \\
\href{https://developers.google.com/earth-engine/datasets/catalog/DOE_ORNL_LandScan_HD_Ukraine_202201}{LandScan}
& 2013--Present & 100m & Ukraine \\
\bottomrule()
\end{longtable}

\hypertarget{building-footprints}{%
\section{Building Footprints}\label{building-footprints}}

\begin{figure}

{\centering \includegraphics{././images/footprints.png}

}

\caption{Building footprints in Mariupol, Ukraine colored by whether the
building is damaged}

\end{figure}

A building footprint dataset contains the two dimensional outlines of
buildings in a given area. Currently, GEE hosts one building footprint
dataset which covers all of Africa. In 2022, Microsoft released a free
\href{https://www.microsoft.com/en-us/maps/building-footprints}{global
building footprint dataset}, though to use it in Earth Engine you'll
have to download it from their
\href{https://github.com/Microsoft/USBuildingFootprints}{GitHub page}
and upload it manually to GEE. The same goes for OpenStreetMap (OSM), a
public database of building footprints, roads, and other features that
also contains useful annotations for many buildings indicating their
use. \href{https://www.youtube.com/watch?v=bJkV3l5Haq0}{Benjamin Strick}
has a great youtube video on conducting investigations using OSM data.

\hypertarget{applications-7}{%
\subsection*{Applications:}\label{applications-7}}
\addcontentsline{toc}{subsection}{Applications:}

\begin{itemize}
\tightlist
\item
  Joining damage estimate data with the number of buildings in an area
\end{itemize}

\hypertarget{datasets-7}{%
\subsection*{Datasets}\label{datasets-7}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3548}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3871}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.2581}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Dataset
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/GOOGLE_Research_open-buildings_v2_polygons}{Open
Buildings} & 2022 & Africa \\
\bottomrule()
\end{longtable}

\hypertarget{administrative-boundaries}{%
\section{Administrative Boundaries}\label{administrative-boundaries}}

\begin{figure}

{\centering \includegraphics{././images/fao_gaul.jpg}

}

\caption{Second-level administrative boundaries in Yemen}

\end{figure}

Spatial analysis often have to aggregate information over a defined
area; we may want to assess the total burned area by province in
Ukraine, or count the number of Saudi airstrikes by district in Yemen.
For that, we need data on these administrative boundaries. GEE hosts
several such datasets at the country, province, and district (or
equivalent) level.

\hypertarget{applications-8}{%
\subsection*{Applications}\label{applications-8}}
\addcontentsline{toc}{subsection}{Applications}

\begin{itemize}
\tightlist
\item
  Quick spatial calculations for different provinces/districts in a
  country

  \begin{itemize}
  \tightlist
  \item
    e.g.~counts of conflict events by district over time
  \end{itemize}
\end{itemize}

\hypertarget{datasets-8}{%
\subsection*{Datasets}\label{datasets-8}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3548}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3871}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.2581}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Dataset
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/tags/gaul}{FAO
GAUL} & 2015 & Global \\
\bottomrule()
\end{longtable}

\hypertarget{global-power-plant-database}{%
\section{Global Power Plant
Database}\label{global-power-plant-database}}

\begin{figure}

{\centering \includegraphics{././images/power.jpg}

}

\caption{Power plants in Ukraine colored by type}

\end{figure}

The Global Power Plant Database is a comprehensive, open source database
of power plants around the world. It centralizes power plant data to
make it easier to navigate, compare and draw insights. Each power plant
is geolocated and entries contain information on plant capacity,
generation, ownership, and fuel type. As of June 2018, the database
includes around 28,500 power plants from 164 countries. The database is
curated by the
\href{https://datasets.wri.org/dataset/globalpowerplantdatabase}{World
Resources Institude (WRI)}.

\hypertarget{applications-9}{%
\subsection*{Applications:}\label{applications-9}}
\addcontentsline{toc}{subsection}{Applications:}

\begin{itemize}
\tightlist
\item
  Analyzing the impact of conflict on critical infrastructure.

  \begin{itemize}
  \tightlist
  \item
    e.g.~fighting in Ukraine taking place around nuclear power
    facilities.
  \end{itemize}
\item
  Could be combined with the atmospheric measurements of different
  pollutants and the population estimates data to assess the impact of
  various forms of energy generation on air quality and public health.
\end{itemize}

\hypertarget{datasets-9}{%
\subsection*{Datasets}\label{datasets-9}}
\addcontentsline{toc}{subsection}{Datasets}

\begin{longtable}[]{@{}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3548}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.3871}}
  >{\raggedright\arraybackslash}p{(\columnwidth - 4\tabcolsep) * \real{0.2581}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\raggedright
Dataset
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Timeframe
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Coverage
\end{minipage} \\
\midrule()
\endhead
\href{https://developers.google.com/earth-engine/datasets/catalog/WRI_GPPD_power_plants}{GPPD}
& 2018 & Global \\
\bottomrule()
\end{longtable}

\hypertarget{getting-started}{%
\chapter{Getting Started}\label{getting-started}}

\hypertarget{programming-basics}{%
\section{Programming Basics}\label{programming-basics}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author}{%
\subsection*{Author}\label{author}}

Ujaval Gandhi

\hypertarget{overview}{%
\subsection*{Overview}\label{overview}}

This chapter introduces the Google Earth Engine application programming
interface (API) and the JavaScript syntax needed to use it. You will
learn about the Code Editor environment and get comfortable typing,
running, and saving scripts. You will also learn the basics of
JavaScript language, such as variables, data structures, and functions.

\hypertarget{learning-outcomes}{%
\subsection*{Learning Outcomes}\label{learning-outcomes}}

\begin{itemize}
\tightlist
\item
  Familiarity with the Earth Engine Code Editor.
\item
  Familiarity with the JavaScript syntax.
\item
  Ability to use the Earth Engine API functions from the Code Editor.
\end{itemize}

\hypertarget{assumes-you-know-how-to}{%
\subsection*{Assumes you know how to:}\label{assumes-you-know-how-to}}

\begin{itemize}
\tightlist
\item
  Sign up for an Earth Engine account (See the Google documentation for
  details).
\item
  Access the Earth Engine Code Editor (See the Google documentation for
  details).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-1}{%
\subsection*{Introduction}\label{introduction-1}}

In order to use Earth Engine well, you will need to develop basic skills
in remote sensing and programming. The language of this book is
JavaScript, and you will begin by learning how to manipulate variables
using it. With that base, you'll learn about viewing individual
satellite images, viewing collections of images in Earth Engine, and how
common remote sensing terms are referenced and used in Earth Engine.

Google Earth Engine is a cloud-based platform for scientific data
analysis. It provides ready-to-use, cloud-hosted datasets and a large
pool of servers. One feature that makes Earth Engine particularly
attractive is the ability to run large computations very fast by
distributing them across a large pool of servers. The ability to
efficiently use cloud-hosted datasets and computation is enabled by the
Earth Engine API.

An API is a way to communicate with Earth Engine servers. It allows you
to specify what computation you would like to do, and then to receive
the results. The API is designed so that users do not need to worry
about how~the computation is distributed across a cluster of machines
and the results are assembled. Users of the API simply specify what
needs to be done. This greatly simplifies the code by hiding the
implementation detail from the users. It also makes Earth Engine very
approachable for users who are not familiar with writing code.

The Earth Engine platform comes with a web-based Code Editor that allows
you to start using the Earth Engine JavaScript API without any
installation. It also provides additional functionality to display your
results on a map, save your scripts, access documentation, manage tasks,
and more. It has a one-click mechanism to share your code with other
users---allowing for easy reproducibility and collaboration. In
addition, the JavaScript API comes with a user interface library, which
allows you to create charts and web-based applications with little
effort.

\hypertarget{getting-started-in-the-code-editor}{%
\subsection{Getting Started in~the Code
Editor}\label{getting-started-in-the-code-editor}}

If you have not already done so, be sure to add the book's code
repository to the Code Editor by
entering~\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1670414092101269\&usg=AOvVaw2sJyDO_fhq1tcjG77pri7V}{}\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1670414092101852\&usg=AOvVaw088kfXu4o_Mp4b8DJBPYjH}{https://code.earthengine.google.com/?accept\_repo=projects/gee-edu/book}~into
your browser. The book's scripts will then be available in the script
manager panel. If you have trouble finding the repo, you can visit
\href{https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit\%23slide\%3Did.g18a7b4b055d_0_624\&sa=D\&source=editors\&ust=1670414092102526\&usg=AOvVaw3ZCmkCOjrZEWqxfjRZPOCn}{this
link}~for help.

The Code Editor is an integrated development~environment for the Earth
Engine JavaScript API. It offers an easy way to type, debug, run, and
manage code. Once you have followed Google's documentation on
registering for an Earth Engine account, you should follow the
documentation to open the Code Editor. When you first visit the Code
Editor, you will see a screen such as the one shown in Fig. F1.0.1.

\begin{figure}

{\centering \includegraphics{./F1/image41.png}

}

\caption{Fig. F1.0.1~The Earth Engine Code Editor}

\end{figure}

The Code Editor (Fig. F1.0.1) allows you to type JavaScript code and
execute it. When you are first learning a new language and getting used
to a new programming environment, it is customary to make a program to
display the words ``Hello World.'' This is a fun way to start coding
that shows you how to give input to the program and how to execute it.
It also shows where the program displays the output. Doing this in
JavaScript is quite simple. Copy the following~code into the center
panel.

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Hello World\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The line of code above uses the JavaScript print~function to print the
text ``Hello World'' to the screen. Once you enter the code, click the
Run~button. The output will be displayed on the upper right-hand panel
under the Console~tab (Fig. F1.0.2.).

\begin{figure}

{\centering \includegraphics{./F1/image57.png}

}

\caption{Fig. F1.0.2~Typing and running code}

\end{figure}

You now know where to type your code, how to run it, and where to look
for the output. You just wrote your first Earth Engine script and may
want to save it. Click the Save~button (Fig. F1.0.3).

\begin{figure}

{\centering \includegraphics{./F1/image5.png}

}

\caption{Fig. F1.0.3~Saving a script}

\end{figure}

If this is your first time using the Code Editor, you will be prompted
to create a home folder. This is a folder in the cloud where all your
code will be saved. You can pick a name of your choice, but remember
that it cannot be changed and will forever be associated with your
account. A good choice for the name would be your Google Account
username (Fig. F1.0.4).

\begin{figure}

{\centering \includegraphics{./F1/image64.png}

}

\caption{Fig. F1.0.4~Creating a home folder}

\end{figure}

Once your home folder is created, you will be prompted to enter a
new~repository. A repository can help you organize and share code. Your
account can have multiple repositories and each repository can have
multiple scripts inside it. To get started, you can create a repository
named ``default'' (Fig. F1.0.5).

\begin{figure}

{\centering \includegraphics{./F1/image33.png}

}

\caption{Fig. F1.0.5~Creating a new repository}

\end{figure}

Finally, you will be able to save your script inside the newly created
repository. Enter the name ``hello\_world'' and click OK~(Fig. F1.0.6).

\begin{figure}

{\centering \includegraphics{./F1/image37.png}

}

\caption{Fig. F1.0.6~Saving a file}

\end{figure}

Once the script is saved, it will appear in the script manager panel
(Fig. F1.0.7). The scripts are saved in the cloud and will always be
available to you when you open the Code Editor.

\begin{figure}

{\centering \includegraphics{./F1/image24.png}

}

\caption{Fig. F1.0.7~The script manager}

\end{figure}

Now you should be familiar with how to create, run, and save your
scripts in the Code Editor. You are ready to start learning the basics
of JavaScript.

\hypertarget{javascript-basics}{%
\subsection{JavaScript Basics}\label{javascript-basics}}

To be able to construct a script for your analysis, you will need to use
JavaScript. This section covers the JavaScript syntax and basic data
structures. In the sections that follow, you will see more JavaScript
code, noted in a distinct font and with shaded background. As you
encounter code,~paste it into the Code Editor and run the script.

\hypertarget{variables}{%
\subsubsection*{Variables}\label{variables}}
\addcontentsline{toc}{subsubsection}{Variables}

In a programming language, variables are used to store data values. In
JavaScript, a variable is defined using the var~keyword followed by the
name of the variable. The code below assigns the text ``San Francisco''
to the variable named~city. Note that the text string in the code should
be surrounded by quotes. You are free to use either '~(single quotes) or
``~(double quotes), and they must match at the beginning and end of each
string. In your programs, it is advisable to be consistent---use either
single quotes or double quotes throughout a given script~(the code in
this book generally uses single quotes for code).~Each statement of your
script should typically end with a semicolon, although Earth Engine's
code editor does not require it.~

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var city }\OperatorTok{=} \StringTok{\textquotesingle{}San Francisco\textquotesingle{}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If you print the variable city, you will get the value stored in the
variable (San Francisco) printed in the Console.~

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{print}\NormalTok{(city)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

When you assign a text value, the variable is automatically assigned the
type string. You can also assign numbers to variables and create
variables of type number. The following code creates a new variable
called population and assigns a number as its value.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var population }\OperatorTok{=} \DecValTok{873965}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(population)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\hypertarget{lists}{%
\subsubsection*{Lists}\label{lists}}
\addcontentsline{toc}{subsubsection}{Lists}

It is helpful to be able to store multiple values in a single variable.
JavaScript provides a data structure called a list~that can hold
multiple values. We can create a new list using the square brackets
{[}{]}~and adding multiple values separated by a comma.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var cities }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}San Francisco\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Los Angeles\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}New York\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Atlanta\textquotesingle{}}\NormalTok{]}\OperatorTok{;} 
\FunctionTok{print}\NormalTok{(cities)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If you look at the output in the Console, you will see ``List'' with an
expander arrow (▹) next to it. Clicking on the arrow will expand the
list and show you its content. You will notice that along with the four
items in the list, there is a number next to each value. This is the
index~of each item. It allows you to refer to each item in the list
using a numeric value that indicates its position in the list.

\begin{figure}

{\centering \includegraphics{./F1/image10.png}

}

\caption{Fig. F1.0.8~A JavaScript list}

\end{figure}

\hypertarget{objects}{%
\subsubsection*{Objects}\label{objects}}
\addcontentsline{toc}{subsubsection}{Objects}

Lists allow you to store multiple values in a single~container variable.
While useful, it is not appropriate to store structured data. It is
helpful to be able to refer to each item with its name rather than its
position. Objects in JavaScript allow you to store key-value pairs,
where each value can be referred to by its key. You can create a
dictionary using the curly braces \{\}. The code below creates an object
called cityData~with some information about San Francisco.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var cityData }\OperatorTok{=}\NormalTok{ \{   }
    \StringTok{\textquotesingle{}city\textquotesingle{}}\OperatorTok{:} \StringTok{\textquotesingle{}San Francisco\textquotesingle{}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}coordinates\textquotesingle{}}\OperatorTok{:}\NormalTok{ [}\OperatorTok{{-}}\FloatTok{122.4194}\OperatorTok{,} \FloatTok{37.7749}\NormalTok{]}\OperatorTok{,}   
    \StringTok{\textquotesingle{}population\textquotesingle{}}\OperatorTok{:} \DecValTok{873965} 
\NormalTok{    \}}\OperatorTok{;}  

\FunctionTok{print}\NormalTok{(cityData)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We can use multiple lines to define the object. Only when we put in the
semicolon (;) is the command considered complete. The object will be
printed in the Console. You can see that instead of a numeric index,
each item has a label. This is known as the key~and can be used to
retrieve the value of an item.

\begin{figure}

{\centering \includegraphics{./F1/image40.png}

}

\caption{Fig. F1.0.9~A JavaScript object}

\end{figure}

\hypertarget{functions}{%
\subsubsection*{Functions}\label{functions}}
\addcontentsline{toc}{subsubsection}{Functions}

While using Earth Engine, you will need to define your own functions.
Functions take user inputs, use them to carry out some computation, and
send an output back. Functions allow you to group a set of operations
together and repeat the same operations with different parameters
without having to rewrite them every time. Functions are defined using
the \texttt{function}~keyword. The code below defines a function called
greet~that takes an input called name~and returns a greeting with
Hello~prefixed to it. Note that we can call the function with different
input and it generates different outputs with the same code.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var greet }\OperatorTok{=} \KeywordTok{function}\NormalTok{(name) \{   }
\NormalTok{    return }\StringTok{\textquotesingle{}Hello \textquotesingle{}} \OperatorTok{+}\NormalTok{ name}\OperatorTok{;}  
\NormalTok{    \}}\OperatorTok{;}
    
\FunctionTok{print}\NormalTok{(}\FunctionTok{greet}\NormalTok{(}\StringTok{\textquotesingle{}World\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\FunctionTok{greet}\NormalTok{(}\StringTok{\textquotesingle{}Readers\textquotesingle{}}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F1/image54.png}

}

\caption{Fig. F1.0.10~JavaScript function output}

\end{figure}

\hypertarget{comments}{%
\subsubsection{Comments}\label{comments}}

While writing code, it is useful to add a bit of text to explain the
code or leave a note for yourself. It is a good programming practice to
always add comments in the code explaining each step. In JavaScript, you
can prefix any line with two forward slashes // to make it a comment.
The text in the comment will be ignored by the interpreter and will not
be executed.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// This is a comment!}
\end{Highlighting}
\end{Shaded}

Congratulations! You have learned enough JavaScript to be able to use
the Earth Engine API. In the next section, you will see how to access
and execute Earth Engine API functions using JavaScript.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F10a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{earth-engine-api-basics}{%
\subsection{Earth Engine API Basics}\label{earth-engine-api-basics}}

The Earth Engine API is vast and provides objects and methods to do
everything from simple math to advanced algorithms for image processing.
In the Code Editor, you can switch to the Docs tab to see the API
functions grouped by object types. The API functions have the prefix
ee~(for Earth Engine).

\begin{figure}

{\centering \includegraphics{./F1/image59.png}

}

\caption{Fig. F1.0.12~Earth Engine API docs}

\end{figure}

Let's learn to use the API. Suppose~you want to add two numbers,
represented by the variables a~and b, as below. Make a new script and
enter the following:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var a }\OperatorTok{=} \DecValTok{1}\OperatorTok{;}  
\NormalTok{var b }\OperatorTok{=} \DecValTok{2}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In Sect. 1, you learned how to store numbers in variables, but not how
to do any computation. This is because when you use Earth Engine, you do
not do addition using JavaScript operators. For example, you would not
write ``var c = a + b'' to add the two numbers. Instead, the Earth
Engine API provides you with functions to do this, and it is important
that you use the API functions whenever you can. It may seem awkward at
first, but using the functions, as we'll describe below, will help you
avoid timeouts and create efficient code.

Looking at the Docs~tab, you will find a group of methods that can be
called on an ee.Number. Expand it to see the various functions available
to work with numbers. You will see the ee.Number~function that creates
an Earth Engine number object from a value. In the list of functions,
there is an add~function for adding two numbers. That's what you use to
add~a~and b.

\begin{figure}

{\centering \includegraphics{./F1/image13.png}

}

\caption{Fig. F1.0.13~ee.Number~module}

\end{figure}

To~add~a~and b, we first create an ee.Number~object from variable~a~with
ee.Number(a). And then we can use the add(b)~call to add the value
of~b~to it. The following code shows the syntax and prints the
result~which, of course, is the value 3.~

var~result = ee.Number(a).add(b);\\
print(result);

By now you may have realized that when learning to program in Earth
Engine, you do not need to deeply learn JavaScript or Python---instead,
they are ways to access the Earth Engine API. This API is the same
whether it is called from JavaScript or Python.

Here's another example to drive this point home. Let's say you are
working on a task that requires you to create a list of years from 1980
to 2020 with a five-year interval. If you are faced with this task, the
first step is to switch to the Docs tab and open the ee.List~module.
Browse through the functions and see if there are any functions that can
help. You will notice a function ee.List.sequence. Clicking on it will
bring up the documentation of the function.

\begin{figure}

{\centering \includegraphics{./F1/image65.png}

}

\caption{Fig. F1.0.14~The ee.List.sequence~function}

\end{figure}

The function ee.List.sequence~is able to generate a sequence of numbers
from a given start~value to the end~value. It also has an optional
parameter step~to indicate the increment between each number. We can
create a ee.List~of numbers representing years from 1980 to 2020,
counting by 5, by calling this predefined function with the following
values: start = 1980, end = 2020, and step = 5.

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var yearList }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{List}\OperatorTok{.}\FunctionTok{sequence}\NormalTok{(}\DecValTok{1980}\OperatorTok{,} \DecValTok{2020}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(yearList)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The output printed in the Console~will show that the variable
yearList~indeed contains the list of years with the correct interval.

\begin{figure}

{\centering \includegraphics{./F1/image29.png}

}

\caption{Fig. F1.0.15~Output of ee.List.sequence~function}

\end{figure}

You just accomplished a moderately complex programming task with the
help of Earth Engine API.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F10b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion}{%
\subsection*{Conclusion}\label{conclusion}}
\addcontentsline{toc}{subsection}{Conclusion}

This chapter introduced the Earth Engine API. You also learned the
basics of JavaScript syntax to be able to use the API in the Code Editor
environment. We hope you now feel a bit more comfortable starting your
journey to become an Earth Engine developer. Regardless of your
programming background or familiarity with JavaScript, you have the
tools at your disposal to start using the Earth Engine API to build
scripts for remote sensing analysis.

\hypertarget{exploring-images}{%
\section{Exploring Images}\label{exploring-images}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-1}{%
\subsection*{Author}\label{author-1}}

Jeff Howarth

\hypertarget{overview-1}{%
\subsection*{Overview}\label{overview-1}}

Satellite images are at the heart of Google Earth Engine's power. This
chapter teaches you how to inspect and visualize data stored in image
bands. We first visualize individual bands as separate map layers and
then explore a method to visualize three different bands in a single
composite layer. We compare different kinds of composites for satellite
bands that measure electromagnetic radiation in the visible and
non-visible spectrum. We then explore images that represent more
abstract attributes of locations, and create a composite layer to
visualize change over time. ~

\hypertarget{learning-outcomes-1}{%
\subsection*{Learning Outcomes}\label{learning-outcomes-1}}

\begin{itemize}
\tightlist
\item
  Using the Code Editor to load an image
\item
  Using code to select image bands and visualize them as map layers
\item
  Understanding true- and false-color composites of images
\item
  Constructing new multiband images.
\item
  Understanding how additive color works and how to interpret RGB
  composites.
\end{itemize}

\hypertarget{assumes-you-know-how-to-1}{%
\subsection*{Assumes you know how to:}\label{assumes-you-know-how-to-1}}

\begin{itemize}
\tightlist
\item
  Sign up for an Earth Engine account, open the Code Editor, and
  save~your script (Chap. F1.0).
\end{itemize}

\end{tcolorbox}

\hypertarget{accessing-an-image}{%
\subsection{Accessing~an Image}\label{accessing-an-image}}

If you have not already done so, be sure to add the book's code
repository to the Code Editor by
entering~\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1670414092189999\&usg=AOvVaw1jWHeBmeq93I_lo_9useCA}{}\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1670414092190705\&usg=AOvVaw3Z7cK8r6eOSYUceNjA8oUg}{https://code.earthengine.google.com/?accept\_repo=projects/gee-edu/book}~into
your browser. The book's scripts will then be available in the script
manager panel. If you have trouble finding the repo, you can visit
\href{https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit\%23slide\%3Did.g18a7b4b055d_0_624\&sa=D\&source=editors\&ust=1670414092191415\&usg=AOvVaw2eETuRpR5worezkj7citx6}{this
link}~for help.

To begin, you will construct an image with the Code Editor. In the
sections that follow, you will see code in a distinct font and with
shaded background. As you encounter code, paste it into the center panel
of the Code Editor and click Run.

First, copy and paste the following:

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var first\_image }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LT05/C02/T1\_L2/LT05\_118038\_20000606\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

When you click Run, Earth Engine will load an image captured by the
Landsat 5 satellite on June 6, 2000. You will not yet see any output.

You can explore the image in several ways. To start, you can retrieve
metadata~(descriptive data about the image) by printing the image to the
Code Editor's Console~panel:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{print}\NormalTok{(first\_image)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the Console~panel, you may need to click the expander arrows to show
the information. You should be able to read that this image consists of
19 different bands. For each band, the metadata lists four properties,
but for now let's simply note that the first property is a name~or label
for the band enclosed in quotation marks. For example, the name of the
first band is ``SR\_B1'' (Fig. F1.1.1).

\begin{figure}

{\centering \includegraphics{./F1/image66.png}

}

\caption{Fig. F1.1.1~Image metadata printed to Console~panel}

\end{figure}

A satellite~sensor like Landsat 5 measures the magnitude of radiation in
different portions of the electromagnetic spectrum. The first six bands
in our image (``SR\_B1'' through ``SR\_B7'') contain measurements for
six different portions of the spectrum. The first three bands measure
visible portions of the spectrum, or quantities of blue, green, and red
light. The other three bands measure infrared portions of the spectrum
that are not visible to the human eye.

An image band is an example of a raster data model, a method of storing
geographic data in a two-dimensional grid of pixels, or picture
elements.

\hypertarget{visualizing-an-image}{%
\subsection{Visualizing an Image}\label{visualizing-an-image}}

Now let's add one of the bands to the map as a layer~so that we can see
it. ~

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(first\_image}\OperatorTok{,} \CommentTok{//  dataset to display   }
\NormalTok{        \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\NormalTok{]}\OperatorTok{,} \CommentTok{//  band to display       }
        \DataTypeTok{min}\OperatorTok{:} \DecValTok{8000}\OperatorTok{,} \CommentTok{//  display range         }
        \DataTypeTok{max}\OperatorTok{:} \DecValTok{17000}\NormalTok{\}}\OperatorTok{,}   
        \StringTok{\textquotesingle{}Layer 1\textquotesingle{}} \CommentTok{//  name to show in Layer Manager  }
\NormalTok{        )}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The code here uses the addLayer~method of the map~in the Code Editor.
There are four important components of the command above:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
  first\_image: This is the dataset to display on the map.
\item
  bands: These are the particular bands from the dataset to display on
  the map. In our example, we displayed a single band named ``SR\_B1''.
\item
  min, max: These represent the lower and upper bounds of values
  from~``SR\_B1'' to display on the screen. By default, the minimum
  value provided (8000) is mapped to black, and the maximum value
  provided (17000) is mapped to white. The values between the minimum
  and maximum are mapped linearly to grayscale between black and white.
  Values below 8000 are drawn as black. Values above 17000 are drawn as
  white. Together, the bands, min, and max parameters define
  visualization parameters, or instructions for data display.
\item
  `Layer 1': This is a label for the map layer to display in the Layer
  Manager. This label appears in the dropdown menu of layers in the
  upper right of the map.
\end{enumerate}

When you run the code, you might not notice the image displayed unless
you pan around and look for it. To do this, click and drag the map
towards Shanghai, China. (You can also jump there by typing ``Shanghai''
into the Search panel at the top of the Code Editor, where the prompt
says Search places and datasets\ldots) Over Shanghai, you should see a
small, dark, slightly angled square. Use the zoom~tool (the +~sign,
upper left of map) to increase the zoom level and make the square appear
larger. ~

Can you recognize any features in the image? By comparing it to the
standard Google map that appears under the image (as the base layer),
you should be able to distinguish the coastline. The water near the
shore generally appears a little lighter than the land, except perhaps
for a large, light-colored blob on the land in the bottom of the image.

Let's explore this image with the Inspector tool. When you click on the
Inspector~tab~on the right side of the Code Editor (Fig. F1.1.2, area
A),~your cursor should now look like crosshairs. When you click on a
location in the image, the Inspector~panel will report data for that
location under three categories as follows: ~

\begin{figure}

{\centering \includegraphics{./F1/image69.png}

}

\caption{Fig. F1.1.2~Image data reported through the Inspector~panel}

\end{figure}

\begin{itemize}
\item
  Point: data about the location on the map. This includes the
  geographic location (longitude and latitude) and some data about the
  map display (zoom level and scale).
\item
  Pixels: data about the pixel in the layer. If you expand this, you
  will see the name of the map layer, a description of the data source,
  and a bar chart. In our example, we see ``Layer 1'' is drawn from an
  image dataset that contains 19 bands. Under the layer name, the chart
  displays the pixel value at the location that you clicked for each
  band in the dataset. When you hover your cursor over a bar, a panel
  will pop up to display the band name and ``band value'' (pixel value).
  To find the pixel value for ``SR\_B1'', hover the cursor over the
  first bar on the left. Alternatively, by clicking on the little blue
  icon to the right of ``Layer 1'' (Fig. F1.1.2, area B), you will
  change the display from a bar chart to a dictionary that reports the
  pixel value for each band. ~ ~
\item
  Objects: data about the source dataset. Here you will find metadata
  about the image that looks very similar to what you retrieved earlier
  when you directed Earth Engine to print the image to the Console. ~
\end{itemize}

Let's add two more bands to the map.

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   first\_image}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{8000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{17000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}Layer 2\textquotesingle{}}\OperatorTok{,}   
    \DecValTok{0}\OperatorTok{,} \CommentTok{//  shown   }
    \DecValTok{1}\ErrorTok{ }\CommentTok{//  opacity  }
\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   first\_image}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{8000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{17000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}Layer 3\textquotesingle{}}\OperatorTok{,}   
    \DecValTok{1}\OperatorTok{,} \CommentTok{//  shown   }
    \DecValTok{0}\ErrorTok{ }\CommentTok{//  opacity  }
\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the code above, notice that we included two additional parameters to
the Map.addLayer~call. One~parameter controls whether or not the layer
is shown~on the screen when the layer is drawn. It may be either 1
(shown) or 0 (not shown). The other parameter defines the opacity~of the
layer, or your ability to ``see through'' the map layer. The opacity
value can range between 0 (transparent) and 1 (opaque).

\begin{figure}

{\centering \includegraphics{./F1/image36.png}

}

\caption{Fig. F1.1.3~Three bands from the Landsat image, drawn as three
different grayscale layers}

\end{figure}

Do you see how these new parameters influence the map layer displays
(Fig. F1.1.3)? For Layer 2, we set the shown parameter as 0. For Layer
3, we set the opacity parameter as 0. As a result, neither layer is
visible to us when we first run the code. We can make each layer visible
with controls in the Layers~manager checklist~on the map (at top right).
Expand~this list and you should see the names that we gave each layer
when we added them to the map. Each name sits between a checkbox and an
opacity slider. To make Layer 2 visible, click the checkbox (Fig.
F1.1.3, area A). To make Layer 3 visible, move the opacity slider to the
right (Fig. F1.1.3, area B).

By manipulating these controls, you should notice that these layers are
displayed as a stack, meaning one on top of the other. For example, set
the opacity for each layer to be 1 by pushing the opacity sliders all
the way to the right. Then make sure each box is checked next to each
layer so that all the layers are shown. Now you can identify which layer
is on top of the stack by checking and unchecking each layer. If a layer
is on top of another, unchecking the top layer will reveal the layer
underneath. If a layer is under another layer in the stack, then
unchecking the bottom layer will not alter the display (because the top
layer will remain visible). If you try this on our stack, you should see
that the list order reflects the stack order, meaning that the layer at
the top of the layer list appears on the top of the stack.~Now compare
the order of the layers in the list to the sequence of operations in
your script.~What layer did your script add first and where does this
appear in the layering order on the map?

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F11a.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{true-color-composites}{%
\subsection{True-Color Composites}\label{true-color-composites}}

Using the controls~in the Layers manager, explore these layers and
examine how the pixel values in each band differ. Does Layer 2
(displaying pixel values from the ``SR\_B2'' band) appear generally
brighter than Layer 1 (the ``SR\_B1'' band)? Compared with Layer 2, do
the ocean waters in Layer 3 (the ``SR\_B3'' band) appear a little darker
in the north, but a little lighter in the south? ~

We can use color to compare these visual differences in the pixel values
of each band layer all at once as an RGB composite. This method uses the
three primary colors (red, green, and blue) to display each pixel's
values across three bands.

To try this, add this code and run it.

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   first\_image}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{8000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{17000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}Natural Color\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result (Fig. F1.1.4) looks like the world we see, and is referred to
as a natural-color composite,~because it naturally pairs the spectral
ranges of the image bands to display colors. Also called a true-color
composite, this image shows the~red spectral band with shades of red,
the green band with shades of green, and the blue band with shades of
blue. We specified the pairing simply through the order of the bands in
the list: B3, B2, B1. Because bands 3, 2, and 1 of Landsat 5 correspond
to the real-world colors of red, green, and blue, the image resembles
the world that we would see outside the window of a plane or with a
low-flying drone. ~

\begin{figure}

{\centering \includegraphics{./F1/image39.png}

}

\caption{Fig. F1.1.4 True-color composite}

\end{figure}

\hypertarget{false-color-composites}{%
\subsection{False-Color Composites}\label{false-color-composites}}

As you saw when you~printed the band list~(Fig. F1.1.1), a Landsat image
contains many more bands than just the three true-color bands. We can
make RGB composites to show combinations of any of the bands---even
those outside what the human eye can see. For example, band 4 represents
the near-infrared band, just outside the range of human vision. Because
of its value in distinguishing environmental conditions, this band was
included on even the earliest 1970s Landsats. It has different values in
coniferous and deciduous forests, for example, and can indicate crop
health. To see an example of this, add this code to your script and run
it. ~

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   first\_image}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{8000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{17000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}False Color\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In this false-color composite (Fig. F1.1.5), the display colors no
longer pair naturally with the bands. This particular example, which is
more precisely referred to as a color-infrared composite,~is a scene
that we could not observe with our eyes, but that you can learn to read
and interpret. Its meaning can be deciphered logically by thinking
through what is passed to the red, green, and blue color channels.

\begin{figure}

{\centering \includegraphics{./F1/image21.png}

}

\caption{Fig. F1.1.5 Color-infrared image (a false-color~composite)}

\end{figure}

Notice how the land on the northern peninsula appears bright red (Fig.
F1.1.5, area A). This is because for that area, the pixel value of the
first band (which is drawing the near-infrared brightness) is much
higher relative to the pixel value of the other two bands. You can check
this by using the Inspector~tool. Try zooming into a part of the image
with a red patch (Fig. F1.1.5, area B) and clicking on a pixel that
appears red. Then expand the ``False Color'' layer in the
Inspector~panel (Fig. F1.1.6, area A), click the blue icon next to the
layer name (Fig. F1.1.6, area B), and read the pixel value for the three
bands of the composite (Fig. F1.1.6, area C). The pixel value for B4
should be much greater than for B3 or B2.~

\begin{figure}

{\centering \includegraphics{./F1/image22.png}

}

\caption{Fig. F1.1.6 Values of B4, B3, B2 bands for a pixel that appears
bright red}

\end{figure}

In the bottom left corner of the image (Fig. F1.1.5, area C), rivers and
lakes appear very dark, which means that the pixel value in all three
bands is low. However, sediment plumes fanning from the river~into the
sea appear with blue and cyan tints (Fig. F1.1.5, area D). If they look
like primary blue, then the pixel value for the second band (B3) is
likely higher than the first (B4) and third (B2) bands. If they appear
more like cyan, an additive color, it means that the pixel values of the
second and third bands are both greater than the first.

In total, the false-color composite provides more contrast than the
true-color image for understanding differences across the scene. This
suggests that other bands might contain more useful information as well.
We saw earlier that our satellite image consisted of 19 bands. Six of
these represent different portions of the electromagnetic spectrum,
including three beyond the visible spectrum, that can be used to make
different false-color composites. Use the code below to explore a
composite that shows shortwave infrared, near infrared, and visible
green (Fig. F1.1.7). ~

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   first\_image}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{8000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{17000}\NormalTok{\}}\OperatorTok{,}   \StringTok{\textquotesingle{}Short wave false color\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F1/image4.png}

}

\caption{Fig. F1.1.7 Shortwave infrared false-color composite}

\end{figure}

To compare the two false-color composites, zoom into the area shown in
the two pictures of Fig. F1.1.8.~You should notice that bright red
locations in the left composite appear bright green in the right
composite. Why do you think that is? Does the image on the right show
new distinctions not seen in the image on the left? If so, what do you
think they are?~

\includegraphics{./F1/image25.png}

\begin{figure}

{\centering \includegraphics{./F1/image8.png}

}

\caption{Fig. F1.1.8~Near-infrared versus shortwave infrared false-color
composites}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F11b.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{attributes-of-locations}{%
\subsection{Attributes of Locations}\label{attributes-of-locations}}

So far, we have explored bands as a method for storing data about slices
of the electromagnetic spectrum that can be measured by satellites. Now
we will work towards applying the additive color system to bands that
store non-optical and more abstract attributes~of geographic locations.
~

To begin, add this code to your script and run it. ~

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ lights93 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}NOAA/DMSP{-}OLS/NIGHTTIME\_LIGHTS/F101993\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Nighttime lights\textquotesingle{}}\OperatorTok{,}\NormalTok{ lights93)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   lights93}\OperatorTok{,}  
\NormalTok{   \{  }
      \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}stable\_lights\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{63}\ErrorTok{ }\NormalTok{  \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Lights\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

This code loads an image of global nighttime lights and adds a new layer
to the map. Please look at the metadata that we printed to the Console
panel. You should see that the image consists of four bands. The code
selects the ``stable\_lights'' band to display as a layer to the map.
The range of values for display (0--63) represent the minimum and
maximum pixel values in this image.~As mentioned earlier, you can find
this range in the Earth Engine Data Datalog or with other Earth Engine
methods. These will be described in more detail in the next few
chapters.

The global nighttime lights image represents the average brightness of
nighttime lights at each pixel for a calendar year. For those of us who
have sat by a window in an airplane as it descends to a destination at
night, the scene may look vaguely familiar. But the image is very much
an abstraction. It provides us a view of the planet that we would never
be able to see from an airplane or even from space. Night blankets the
entire planet in darkness. There are no clouds. In the ``stable lights''
band, there are no ephemeral sources of light. Lightning strikes,
wildfires, and other transient lights have been removed. It is a layer
that aims to answer one question about our planet at one point in time:
In 1993, how bright were Earth's stable, artificial sources of light?

With the zoom~controls on the map, you can zoom out to see the bright
spot of Shanghai, the large blob of Seoul to the north and east, the
darkness of North Korea except for the small dot of Pyongyang, and the
dense strips of lights of Japan and the west coast of Taiwan (Fig.
F1.1.10). ~

\begin{figure}

{\centering \includegraphics{./F1/image34.png}

}

\caption{Fig. F1.1.10 Stable nighttime lights in 1993}

\end{figure}

\hypertarget{abstract-rgb-composites}{%
\subsection{Abstract RGB Composites ~}\label{abstract-rgb-composites}}

Now we can use the additive color system to make an RGB composite that
compares stable nighttime lights at three different slices of time. Add
the code below to your script and run it. ~

\begin{Shaded}
\begin{Highlighting}[]
\NormalTok{var lights03 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}NOAA/DMSP{-}OLS/NIGHTTIME\_LIGHTS/F152003\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}stable\_lights\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}2003\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\NormalTok{var lights13 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}NOAA/DMSP{-}OLS/NIGHTTIME\_LIGHTS/F182013\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}stable\_lights\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}2013\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\NormalTok{var changeImage }\OperatorTok{=}\NormalTok{ lights13}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(lights03)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(lights93}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}stable\_lights\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}1993\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}change image\textquotesingle{}}\OperatorTok{,}\NormalTok{ changeImage)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   changeImage}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
    \DataTypeTok{max}\OperatorTok{:} \DecValTok{63}\NormalTok{\}}\OperatorTok{,}
    \StringTok{\textquotesingle{}Change composite\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

This code does a few things. First, it creates two new images, each
representing a different slice of time. For both, we use the
select~method to select a band (``stable\_lights'') and the
rename~method to change the band name to indicate the year it
represents. ~

Next, the code uses the addBands~method to create a new, three-band
image that we name ``changeImage''. It does this by taking one image
(lights13) as the first band, using another image (lights03) as the
second band, and the lights93 image seen earlier as the third band. The
third band is given the name ``1993'' as it is placed into the image.

Finally, the code prints metadata to the Console~and adds the layer to
the map as an RGB composite using Map.addLayer. If you look at the
printed metadata, you should see under the label ``change image'' that
our image is composed of three bands, with each band named after a year.
You should also notice the order of the bands in the image: 2013, 2003,
1993. This order determines the color channels used to represent each
slice of time in the composite: 2013 as red, 2003 as green, and 1993 as
blue (Fig. F1.1.11).

\begin{figure}

{\centering \includegraphics{./F1/image51.png}

}

\caption{Fig. F1.1.11 RGB composite of stable nighttime lights (2013,
2003, 1993)}

\end{figure}

We can now read the colors displayed on the layer to interpret different
kinds of changes in nighttime lights across the planet over two decades.
Pixels that appear white have high brightness in all three years. You
can use the Inspector~panel to confirm this. Click on the
Inspector~panel to change the cursor to a crosshair and then click on a
pixel that appears white. Look under the Pixel~category of the
Inspector~panel for the ``Change composite'' layer. The pixel value for
each band should be high (at or near 63). ~

Many clumps of white pixels represent urban cores. If you zoom into
Shanghai, you will notice that the periphery of the white-colored core
appears yellowish and the terminal edges appear reddish. Yellow
represents locations that were bright in 2013 and 2003 but dark in 1993.
Red represents locations that appear bright in 2013 but dark in 2003 and
1993. If you zoom out, you will see this gradient of white core to
yellow periphery to red edge occurs around many cities across the
planet, and shows the global pattern of urban sprawl over the 20-year
period.~

When you zoom out from Shanghai, you will likely notice that each map
layer redraws every time you change the zoom level. In order to explore
the change composite layer more efficiently, use the Layer~manager panel
to not show~(uncheck) all of the layers except for ``Change composite.''
Now the map will respond faster when you zoom and pan because it will
only refresh the single displayed shown layer.

In addition to urban change, the layer also shows changes in resource
extraction activities that produce bright lights. Often, these
activities produce lights that are stable over the span of a year (and
therefore included in the ``stable lights'' band), but are not sustained
over the span of a decade or more. For example, in the Korea Strait
(between South Korea and Japan), you can see geographic shifts of
fishing fleets that use bright halogen lights to attract squid and other
sea creatures towards the water surface and into their nets. Bluish
pixels were likely fished more heavily in 1993 and became used less
frequently by 2003, while greenish pixels were likely fished more
heavily in 2003 and less frequently by 2013 (Fig. F1.1.11).

\begin{figure}

{\centering \includegraphics{./F1/image52.png}

}

\caption{Fig. F1.1.12~Large red blobs in North Dakota and Texas from
fossil fuel extraction in specific years}

\end{figure}

Similarly, fossil fuel extraction produces nighttime lights through gas
flaring. If you pan~to North America (Fig. F1.1.12), red blobs in
Alberta and North Dakota and a red swath in southeastern Texas all
represent places where oil and gas extraction were absent in 1993 and
2003 but booming by 2013. Pan over to the Persian Gulf and you will see
changes that look like holiday lights with dots of white, red, green,
and blue appearing near each other; these distinguish stable and
shifting locations of oil production. Blue lights in Syria near the
border with Iraq signify the abandonment of oil fields after 1993 (Fig.
F1.1.13). Pan~further north and you will see another ``holiday lights''
display from oil and gas extraction around Surgut, Russia. In many of
these places, you can check for oil and gas infrastructure by zooming in
to a~colored spot, making the lights layer not visible, and selecting
the Satellite base layer (upper right).

\begin{figure}

{\centering \includegraphics{./F1/image48.png}

}

\caption{Fig. F1.1.13 Nighttime light changes in the Middle East}

\end{figure}

As you explore this image, remember to check your interpretations with
the Inspector~panel by clicking on a pixel and reading the pixel value
for each band. Refer back to the additive color figure to remember how
the color system works. If you practice this, you should be able to read
any RGB composite by knowing how colors relate to the relative pixel
value of each band. This will empower you to employ false-color
composites as a flexible and powerful method to explore and interpret
geographic patterns and changes on Earth's surface.~

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F11c.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-1}{%
\subsection*{Conclusion}\label{conclusion-1}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, we looked at how an image is composed of one or more
bands, where each band stores data about geographic locations as pixel
values. We explored different ways of visualizing these pixel values as
map layers, including a grayscale display of single bands and RGB
composites of three bands. We created natural and false-color composites
that use additive color to display information in visible and
non-visible portions of the spectrum. We examined additive color as a
general system for visualizing pixel values across multiple bands. We
then explored how bands and RGB composites can be used to represent more
abstract phenomena, including different kinds of change over time.

\hypertarget{survey-of-raster-datasets}{%
\section{Survey~of Raster Datasets}\label{survey-of-raster-datasets}}

The previous chapter introduced you to images, one of the core building
blocks of remotely sensed imagery in Earth Engine. In this chapter, we
will expand on this concept of images by introducing image collections.
Image collections in Earth Engine organize many different images into
one larger data storage structure. Image collections include information
about the location, date collected, and other properties of each image,
allowing you to sift through the ImageCollection~for the exact image
characteristics needed for your analysis.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{authors}{%
\subsection*{Authors}\label{authors}}

Andréa Puzzi Nicolau, Karen Dyson, David Saah, Nicholas Clinton

\hypertarget{overview-2}{%
\subsection*{Overview}\label{overview-2}}

The purpose of this chapter is to introduce you to the many types of
collections of images available in Google Earth Engine. These include
sets of individual satellite images, pre-made composites (which merge
multiple individual satellite images into one composite image),
classified land use and land cover (LULC) maps, weather data, and other
types of datasets. If you are new to JavaScript or programming, work
through Chaps. F1.0 and F1.1 first. ~

\hypertarget{learning-outcomes-2}{%
\subsection*{Learning Outcomes}\label{learning-outcomes-2}}

\begin{itemize}
\tightlist
\item
  Accessing and viewing sets of images~in Earth Engine.
\item
  Extracting single scenes from collections of images.
\item
  Applying visualization parameters in Earth Engine to visualize an
  image.
\end{itemize}

\hypertarget{assumes-you-know-how-to-2}{%
\subsection*{Assumes you know how to:}\label{assumes-you-know-how-to-2}}

\begin{itemize}
\tightlist
\item
  Sign up for an Earth Engine account, open the Code Editor, and save
  your script. (Chap. F1.0)
\item
  Locate the Earth Engine Inspector~and Console~tabs and understand
  their purposes~(Chap. F1.0).
\item
  Use the Inspector tab to assess pixel values~(Chap. F1.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{image-collections-an-organized-set-of-images}{%
\subsection{Image Collections: An Organized Set of
Images}\label{image-collections-an-organized-set-of-images}}

There are many different types of image collections~available in Earth
Engine. These include collections of individual satellite images,
pre-made composites that combine multiple images into one blended image,
classified LULC maps, weather data, and other non-optical data sets.
Each one of these is useful for different types of analyses. For
example, one recent study examined the drivers of wildfires in Australia
(Sulova and Jokar~2021). The research team used the European Center for
Medium-Range Weather Forecast Reanalysis (ERA5) dataset produced by the
European Center for Medium-Range Weather Forecasts (ECMWF) and~is freely
available in Earth Engine. We will look at this dataset later in the
chapter.

You saw some of the basic ways to interact with an individual
ee.Image~in the previous chapter. However, depending on how long a
remote sensing platform has been in operation, there may be thousands or
millions of images collected of Earth. In Earth Engine, these are
organized into an ImageCollection, a specialized data type that has
specific operations available in the Earth Engine API. Like individual
images, they can be viewed with~Map.addLayer.

\hypertarget{view-an-image-collection}{%
\subsection{View an Image Collection}\label{view-an-image-collection}}

The Landsat program from NASA and the United States Geological Survey
(USGS) has launched a sequence of Earth observation satellites, named
Landsat 1, 2, etc. Landsats have been returning images since 1972,
making that collection of images the longest continuous satellite-based
observation of the Earth's surface. We will now view images and basic
information about one of the image collections that is still growing:
collections of scenes taken by the Operational Land Imager aboard
Landsat 8, which was launched in 2013. Copy and paste the following code
into the center panel and click Run. While the enormous image catalog is
accessed, it could take a couple of minutes to see the result in the Map
area. If it takes more than a couple of minutes to see the images, try
zooming in to a specific area to speed up the process.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// View an Image Collection  }
\CommentTok{/////  }
  
\CommentTok{// Import the Landsat 8 Raw Collection.  }
\NormalTok{var landsat8 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Print the size of the Landsat 8 dataset.  }
\FunctionTok{print}\NormalTok{(}
    \StringTok{\textquotesingle{}The size of the Landsat 8 image collection is:\textquotesingle{}}\OperatorTok{,} 
\NormalTok{    landsat8}\OperatorTok{.}\FunctionTok{size}\NormalTok{()}
\NormalTok{    )}\OperatorTok{;}  
  
\CommentTok{// Try to print the image collection.  }
\CommentTok{// }\AlertTok{WARNING}\CommentTok{! Running the print code immediately below produces an error because  }
\CommentTok{// the Console can not print more than 5000 elements.  }
\FunctionTok{print}\NormalTok{(landsat8)}\OperatorTok{;}  
  
\CommentTok{// Add the Landsat 8 dataset to the map as a mosaic. The collection is  }
\CommentTok{// already chronologically sorted, so the most recent pixel is displayed.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsat8}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{5000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{15000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}Landsat 8 Image Collection\textquotesingle{}}
\NormalTok{    )}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

First, let's examine the map output~(Fig.~F1.2.1).

\begin{figure}

{\centering \includegraphics{./F1/image18.png}

}

\caption{Fig.~F1.2.1~USGS Landsat 8 Collection~2 Tier 1 Raw Scenes
collection}

\end{figure}

Notice the high amount of cloud cover, and the ``layered'' look. Zoom
out if needed. This is because Earth Engine is drawing each of the
images that make up the ImageCollection~one on top of the other. The
striped look is the result of how the satellite collects imagery. The
overlaps between images and the individual nature of the images mean
that these are not quite ready for analysis; we will address this issue
in future chapters.

Now examine~the printed size on the Console. It will indicate that there
are more than a million images in the dataset (Fig. F1.2.2). If you
return to this lab in the future, the number will be even larger, since
this active collection is continually growing as the satellite gathers
more imagery. For the same reason, Fig. F1.2.1 might look slightly
different on your map because of this.

\begin{figure}

{\centering \includegraphics{./F1/image9.png}

}

\caption{Fig. F1.2.2 ~Size of the entire Landsat 8 collection. Note that
this number is constantly growing.}

\end{figure}

Note that printing the ImageCollection~returned an error message (Fig.
F1.2.3), because calling print~on an ImageCollection~will write the name
of every image in the collection to the Console. This is the result of
an intentional safeguard within Earth Engine. We don't want to see a
million image names printed to the Console!

\begin{figure}

{\centering \includegraphics{./F1/image71.png}

}

\caption{Fig. F1.2.3.~Error encountered when trying to print the names
and information to the screen for too many elements}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F12a.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Edit~your code to comment out the last two code commands you have
written. This will remove the call to Map.addLayer~that drew every
image, and will remove the print~statement that demanded more than 5000
elements. This will speed up your code in subsequent sections. Placing
two forward slashes (//) at the beginning of a line will make it into a
comment, and any commands on that line will not be executed.

\hypertarget{filtering-image-collections}{%
\subsection{Filtering Image
Collections}\label{filtering-image-collections}}

The ImageCollection~data type in Earth Engine has multiple approaches to
filtering, which helps to pinpoint the exact images you want to view or
analyze from the larger collection.

\hypertarget{filter-by-date}{%
\subsubsection{Filter by Date}\label{filter-by-date}}

One of the filters is filterDate, which allows us to narrow down the
date range of the ImageCollection. Copy the following code to the center
panel (paste it after the previous code you had):

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Filter an Image Collection  }
\CommentTok{/////  }
  
\CommentTok{// Filter the collection by date.  }
\NormalTok{var landsatWinter }\OperatorTok{=}\NormalTok{ landsat8}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}12{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}03{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsatWinter}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{5000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{15000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}Winter Landsat 8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}The size of the Winter Landsat 8 image collection is:\textquotesingle{}}\OperatorTok{,}  
\NormalTok{   landsatWinter}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Examine the mapped landsatWinter~(Fig.~F1.2.4). As described in the
previous chaper, the 5000 and the 15000 values in the visualization
parameters of the Map.addLayer~function of the code above refer to the
minimum and maximum of the range of display values.

\begin{figure}

{\centering \includegraphics{./F1/image38.png}

}

\caption{Fig. F1.2.4~Landsat 8 Winter Collection}

\end{figure}

Now look at the size of the winter Landsat 8 collection. The number is
significantly lower than the number of images in the entire collection.
This is the result of filtering the dates to three months in the winter
of 2020--2021.

\hypertarget{filter-by-location}{%
\subsubsection{Filter by Location}\label{filter-by-location}}

A second frequently used filtering tool~is filterBounds. This filter is
based on a location---for example, a point, polygon, or other geometry.
Copy and paste the code below to filter and add to the map the winter
images from the Landsat 8 Image Collection to a point in Minneapolis,
Minnesota, USA. Note below the Map.addLayer~function to add the
pointMN~to the map with an empty dictionary \{\}~for the
visParams~argument.~This only means that we are not specifying
visualization parameters for this element, and it is being added to the
map with the default parameters.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create an Earth Engine Point object.  }
\NormalTok{var pointMN }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{93.79}\OperatorTok{,} \FloatTok{45.05}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Filter the collection by location using the point.  }
\NormalTok{var landsatMN }\OperatorTok{=}\NormalTok{ landsatWinter}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(pointMN)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(}
\NormalTok{    landsatMN}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{5000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{15000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}MN Landsat 8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add the point to the map to see where it is.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(pointMN}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Point MN\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}The size of the Minneapolis Winter Landsat 8 image collection is: \textquotesingle{}}\OperatorTok{,}  
\NormalTok{   landsatMN}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If we uncheck the Winter Landsat 8 layer under Layers, we can see that
only images that intersect our point have been selected~(Fig. F1.2.5).
Zoom in or out as needed. Note the printed size of the Minneapolis
winter collection---we only have seven images.

\begin{figure}

{\centering \includegraphics{./F1/image62.png}

}

\caption{Fig. F1.2.5~Minneapolis Winter Collection filtered by bounds.}

\end{figure}

The first still represents the map without zoom applied. The collection
is shown inside the red circle. The second still represents the map
after zoom was applied to the region. The red arrow indicates the point
(in black) used to filter by bounds.

\hypertarget{selecting-the-first-image}{%
\subsubsection{Selecting the First
Image}\label{selecting-the-first-image}}

The final operation we will explore is the first~function. This selects
the first image in an ImageCollection. This allows us to place a single
image on the screen for inspection. Copy and paste the code below to
select and view the first image of the Minneapolis Winter Landsat 8
Image Collection. In this case, because the images are stored in time
order in the ImageCollection, it will select the earliest image in the
set.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Select the first image in the filtered collection.  }
\NormalTok{var landsatFirst }\OperatorTok{=}\NormalTok{ landsatMN}\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the first image in the filtered collection.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(landsatFirst}\OperatorTok{,} \DecValTok{7}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(}
\NormalTok{    landsatFirst}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{5000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{15000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}First Landsat 8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The~first~command takes~our stack of location-filtered images and
selects the first image. When the layer is added to the Map area, you
can see that only one image is returned---remember to uncheck the other
layers to be able to visualize the full image (Fig. F1.2.6). We used the
Map.centerObject~to center the map on the landsatFirst~image with a zoom
level of 7 (zoom levels go from 0 to 24).

\begin{figure}

{\centering \includegraphics{./F1/image43.png}

}

\caption{Fig. F1.2.6~First Landsat image from the filtered set}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F12b.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Now that we have the tools to examine different image collections, we
will explore~other datasets.

\hypertarget{collections-of-single-images}{%
\subsection{Collections of Single
Images}\label{collections-of-single-images}}

When learning about image collections in the previous section, you
worked with the Landsat 8 raw image dataset. These raw images have some
important corrections already done for you. However, the raw images are
only one of several image collections produced for Landsat 8. The remote
sensing community has developed additional imagery corrections that help
increase the accuracy and consistency of analyses. The results of each
of these different imagery processing paths is stored in a distinct
ImageCollection~in Earth Engine.

Among the most prominent of these is the ImageCollection~meant to
minimize the effect of the atmosphere between Earth's surface and the
satellite. The view from satellites is made imprecise by the need for
light rays to pass through the atmosphere, even on the clearest day.
There are two important ways~the atmosphere obscures a~satellite's
view:~by affecting the amount of sunlight that strikes the Earth, and by
altering electromagnetic energy~on its trip from its reflection at
Earth's surface to the satellite's receptors.

Unraveling~those effects is called atmospheric correction, a highly
complex process whose details are beyond the scope of this book.
Thankfully, in addition to the raw images from the satellite, each image
for Landsat and certain other sensors is~automatically treated with the
most up-to-date atmospheric correction algorithms, producing a product
referred to as a ``surface reflectance'' ImageCollection. The surface
reflectance~estimates the ratio of upward radiance at the Earth's
surface to downward radiance at the Earth's surface, imitating what the
sensor would have seen if it were hovering a few feet~above the ground.
~

Let's examine~one of these datasets meant to minimize the effects of the
atmosphere between Earth's surface and the satellite. Copy and paste the
code below to import and filter the Landsat 8 surface reflectance
data~(landsat8SR) by date and to a point over San Francisco, California,
USA (pointSF). We use the first~function to select the first image---a
single image from March 18, 2014. By printing the landsat8SRimage~image
on the Console, and accessing its metadata (see Chap. F1.1),~we see that
the band names differ from those in the raw image (Fig. F1.2.7). Here,
they have the form ~``SR\_B\emph{''~as in ``Surface Reflectance Band
}'', where *~is the band number. We can also check the date of the image
by looking at the image ``id'' (Fig. F1.2.7). This has the value
``20140318'', a string indicating that the image was from March 18,
2014.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Collections of single images {-} Landsat 8 Surface Reflectance  }
\CommentTok{/////  }
  
\CommentTok{// Create and Earth Engine Point object over San Francisco.  }
\NormalTok{var pointSF }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{122.44}\OperatorTok{,} \FloatTok{37.76}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Import the Landsat 8 Surface Reflectance collection.  }
\NormalTok{var landsat8SR }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Filter the collection and select the first image.  }
\NormalTok{var landsat8SRimage }\OperatorTok{=}\NormalTok{ landsat8SR}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2014{-}03{-}18\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2014{-}03{-}19\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(pointSF)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Landsat 8 Surface Reflectance image\textquotesingle{}}\OperatorTok{,}\NormalTok{ landsat8SRimage)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F1/image30.png}

}

\caption{Fig. F1.2.7~Landsat 8 Surface Reflectance image bands and date}

\end{figure}

Copy and paste the code below to add~this image to the map with adjusted
R,G, and B bands in the ``bands'' parameter for true-color display~(see
previous chapter).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Center map to the first image.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(landsat8SRimage}\OperatorTok{,} \DecValTok{8}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add first image to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsat8SRimage}\OperatorTok{,}  
\NormalTok{   \{}\DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{7000}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{13000}\NormalTok{\}}\OperatorTok{,}   
    \StringTok{\textquotesingle{}Landsat 8 SR\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F1/image15.png}

}

\caption{Fig. F1.2.8~Landsat 8 Surface Reflectance scene from March 18,
2014}

\end{figure}

Compare this image (Fig. F1.2.8)~with the raw Landsat 8 images from the
previous section (Fig. F1.2.6). Zoom in and out and pan the screen as
needed. What do you notice? Save your script but don't start a new
one---we will keep adding code to this script.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F12c.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{modis-monthly-burned-areas}{%
\subsection{MODIS Monthly Burned
Areas}\label{modis-monthly-burned-areas}}

We'll explore two examples of composites made with data from the MODIS
sensors, a pair of sensors aboard the Terra~and Aqua~satellites. On
these complex sensors, different MODIS bands~produce data at different
spatial resolutions. For the visible bands, the lowest common resolution
is 500 m (red and NIR are 250 m).

Some of the MODIS bands have proven useful in determining where fires
are burning and what areas they have burned. A monthly composite product
for burned areas is available in Earth Engine. Copy and paste the code
below.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the MODIS monthly burned areas dataset.  }
\NormalTok{var modisMonthly }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}MODIS/006/MCD64A1\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Filter the dataset to a recent month during fire season.  }
\NormalTok{var modisMonthlyRecent }\OperatorTok{=}\NormalTok{ modisMonthly}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2021{-}08{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add the dataset to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(modisMonthlyRecent}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}MODIS Monthly Burn\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Uncheck the other layers, and then pan and zoom around the map. Areas
that have burned in the past month will show up as red (Fig. F1.2.11).
Can you see where fires burned areas of California, USA? In Southern and
Central Africa? Northern Australia?

\begin{figure}

{\centering \includegraphics{./F1/image19.png}

}

\caption{Fig. F1.2.11.~MODIS Monthly Burn image over California}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F12d.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Save your script and start a new one by refreshing the page.

\hypertarget{methane}{%
\subsection{Methane}\label{methane}}

Satellites can also collect information about the climate, weather, and
various compounds present in the atmosphere. These satellites leverage
portions of the electromagnetic spectrum~and how different objects and
compounds reflect when hit with sunlight in various wavelengths. For
example, methane~(CH4) reflects the 760 nm portion of the spectrum.
Let's take a closer look at a few of these datasets.

The European Space Agency makes available a methane dataset from
Sentinel-5~in Earth Engine. Copy and paste the code below to add to the
map methane data from the first time of collection on November 28, 2018.
We use the select~function (See Chap. F1.1) to select the
methane-specific band of the dataset. We also introduce values for a new
argument for the visualization parameters of Map.addLayer: We use a
color palette~to display a single band of an image in color. Here, we
chose varying colors from black for the minimum value to red for the
maximum value. Values in

between will have the color in the order outlined by the
palette~parameter (a list of string colors: blue, purple, cyan, green,
yellow, red).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Other satellite products  }
\CommentTok{/////  }
  
\CommentTok{// Import a Sentinel{-}5 methane dataset.  }
\NormalTok{var methane }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S5P/OFFL/L3\_CH4\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Filter the methane dataset.  }
\NormalTok{var methane2018 }\OperatorTok{=}\NormalTok{ methane}\OperatorTok{.}\FunctionTok{select}\NormalTok{(       }\StringTok{\textquotesingle{}CH4\_column\_volume\_mixing\_ratio\_dry\_air\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2018{-}11{-}28\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2018{-}11{-}29\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Make a visualization for the methane data.  }
\NormalTok{var methaneVis }\OperatorTok{=}\NormalTok{ \{  }
  \DataTypeTok{ palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}black\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}purple\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}cyan\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\OperatorTok{,}       \StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{   ]}\OperatorTok{,}  
  \DataTypeTok{ min}\OperatorTok{:} \DecValTok{1770}\OperatorTok{,}  
  \DataTypeTok{ max}\OperatorTok{:} \DecValTok{1920}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Center the Map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(methane2018}\OperatorTok{,} \DecValTok{3}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add the methane dataset to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(methane2018}\OperatorTok{,}\NormalTok{ methaneVis}\OperatorTok{,} \StringTok{\textquotesingle{}Methane\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Notice the different levels of methane over the African continent (Fig.
F1.2.12).

\begin{figure}

{\centering \includegraphics{./F1/image56.png}

}

\caption{Fig. F1.2.12.~Methane levels over the African continent on
November 28, 2018}

\end{figure}

\hypertarget{global-forest-change}{%
\subsection{Global~Forest Change}\label{global-forest-change}}

Another useful land cover product that has been pre-classified for you
and is available in Earth Engine is the Global Forest Change~dataset.
This analysis was conducted between 2000 and 2020. Unlike the WorldCover
dataset, this dataset focuses on the percent of tree cover across the
Earth's surface in a base year of 2000, and how that has changed over
time. Copy and paste the code below to visualize the tree cover in 2000.
Note that in the code below we define the visualization parameters as a
variable treeCoverViz~instead of having its calculation done within the
Map.addLayer~function.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the Hansen Global Forest Change dataset.  }
\NormalTok{var globalForest }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(   }\StringTok{\textquotesingle{}UMD/hansen/global\_forest\_change\_2020\_v1\_8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Create a visualization for tree cover in 2000.  }
\NormalTok{var treeCoverViz }\OperatorTok{=}\NormalTok{ \{  }
  \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}treecover2000\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
  \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
  \DataTypeTok{ max}\OperatorTok{:} \DecValTok{100}\OperatorTok{,}  
  \DataTypeTok{ palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}black\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Add the 2000 tree cover image to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(globalForest}\OperatorTok{,}\NormalTok{ treeCoverViz}\OperatorTok{,} \StringTok{\textquotesingle{}Hansen 2000 Tree Cover\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Notice how areas with high tree cover (e.g., the Amazon) are greener and
areas with low tree cover are darker (Fig. F1.2.15). In case you see an
error on the Console~such as ``Cannot read properties of null,'' don't
worry. Sometimes Earth Engine will show these transient errors, but they
won't affect the script in any way.

\begin{figure}

{\centering \includegraphics{./F1/image68.png}

}

\caption{Fig. F1.2.15~Global Forest Change 2000 tree cover layer}

\end{figure}

Copy and paste the code below to visualize the tree cover loss over the
past 20 years.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a visualization for the year of tree loss over the past 20 years.  }
\NormalTok{var treeLossYearViz }\OperatorTok{=}\NormalTok{ \{  }
  \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}lossyear\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
  \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
  \DataTypeTok{ max}\OperatorTok{:} \DecValTok{20}\OperatorTok{,}  
  \DataTypeTok{ palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Add the 2000{-}2020 tree cover loss image to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(globalForest}\OperatorTok{,}\NormalTok{ treeLossYearViz}\OperatorTok{,} \StringTok{\textquotesingle{}2000{-}2020 Year of Loss\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Leave the previous 2000 tree cover layer checked and analyze the loss
layer on top of it---yellow, orange, and red areas (Fig. F1.2.16). Pan
and zoom around the map. Where has there been recent forest loss (which
is shown in red)?

\begin{figure}

{\centering \includegraphics{./F1/image16.png}

}

\caption{Fig. F1.2.16~Global Forest Change 2000--2020 tree cover loss
(yellow-red) and 2000 tree cover (black-green)}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F12f.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Save your script and start a new one.

\hypertarget{digital-elevation-models}{%
\subsection{Digital Elevation Models}\label{digital-elevation-models}}

Digital elevation models (DEMs) use airborne and satellite instruments
to estimate the elevation of each location. Earth Engine has both local
and global DEMs available. One of the global DEMs available is the
NASADEM dataset, a DEM produced from a NASA mission. Copy and paste the
code below to import the dataset and visualize the elevation band.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the NASA DEM Dataset.  }
\NormalTok{var nasaDEM }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}NASA/NASADEM\_HGT/001\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add the elevation layer to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(nasaDEM}\OperatorTok{,}\NormalTok{ \{  }
  \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}elevation\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
  \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
  \DataTypeTok{ max}\OperatorTok{:} \DecValTok{3000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NASA DEM\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Uncheck the population layer and zoom in to examine the patterns of
topography (Fig. F1.2.18). Can you see where a mountain range is
located? Where is a river located? Try changing the minimum and maximum
in order to make these features more visible. Save your script.

\includegraphics{./F1/image61.png}

Fig. F1.2.18.~NASADEM elevation

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F12g.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-2}{%
\subsection*{Conclusion}\label{conclusion-2}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, we introduced image collections in Earth Engine and
learned how to apply multiple types of filters to image collections to
identify multiple or a single image for use. We also explored a few of
the many different image collections available in the Earth Engine Data
Catalog. Understanding how to find, access, and filter image collections
is an important step in learning how to perform spatial analyses in
Earth Engine.

\hypertarget{references}{%
\subsection*{References}\label{references}}
\addcontentsline{toc}{subsection}{References}

Chander G, Huang C, Yang L, et al (2009) Developing consistent Landsat
data sets for large area applications: The MRLC 2001 protocol. IEEE
Geosci Remote Sens Lett 6:777--781.
https://doi.org/10.1109/LGRS.2009.2025244

Chander G, Markham BL, Helder DL (2009) Summary of current radiometric
calibration coefficients for Landsat MSS, TM, ETM+, and EO-1 ALI
sensors. Remote Sens Environ 113:893--903.
https://doi.org/10.1016/j.rse.2009.01.007

Hansen MC, Potapov PV, Moore R, et al (2013) High-resolution global maps
of 21st-century forest cover change. Science 342:850--853.
https://doi.org/10.1126/science.1244693

Sulova A, Arsanjani JJ (2021) Exploratory analysis of driving force of
wildfires in Australia: An application of machine learning within Google
Earth Engine. Remote Sens 13:1--23. https://doi.org/10.3390/rs13010010

\hypertarget{the-remote-sensing-vocabulary}{%
\section{The Remote~Sensing
Vocabulary}\label{the-remote-sensing-vocabulary}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{authors-1}{%
\subsection*{Authors}\label{authors-1}}

Karen Dyson, Andréa Puzzi Nicolau, David Saah, Nicholas Clinton

\hypertarget{overview-3}{%
\subsection*{Overview}\label{overview-3}}

The purpose of this chapter is to introduce some of the principal
characteristics of remotely sensed images and how they can be examined
in Earth Engine. We discuss spatial resolution, temporal resolution, and
spectral resolution, along with how to access important image metadata.
You will be introduced to image data from several sensors aboard various
satellite platforms. At the completion of the chapter, you will be able
to understand the difference between remotely sensed datasets based on
these characteristics, and how to choose an appropriate dataset for your
analysis based on these concepts. ~

\hypertarget{learning-outcomes-3}{%
\subsection*{Learning Outcomes}\label{learning-outcomes-3}}

\begin{itemize}
\tightlist
\item
  Understanding spatial, temporal, and spectral resolution.
\item
  Navigating the Earth Engine Console~to gather information about a
  digital image, including resolution and other data documentation.
\end{itemize}

\hypertarget{assumes-you-know-how-to-3}{%
\subsection*{Assumes you know how to:}\label{assumes-you-know-how-to-3}}

\begin{itemize}
\tightlist
\item
  Navigate among Earth Engine result tabs (Chap. F1.0).
\item
  Visualize images with a variety of false-color band combinations
  (Chap. F1.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-2}{%
\subsection*{Introduction}\label{introduction-2}}

Images and image collections form the basis of many remote sensing
analyses in Earth Engine. There are many different types of satellite
imagery available to use in these analyses, but not every dataset is
appropriate for every analysis. To choose the most appropriate dataset
for your analysis, you should consider multiple factors. Among these are
the resolution of the dataset---including the spatial, temporal, and
spectral resolutions---as well as how the dataset was created and its
quality.

\hypertarget{searching-for-and-viewing-image-collection-information}{%
\subsection{Searching~for and Viewing Image Collection
Information}\label{searching-for-and-viewing-image-collection-information}}

Earth Engine's search bar can be used to find imagery and to locate
important information about datasets in Earth Engine. Let's use the
search bar, located above the Earth Engine code, to find out information
about the Landsat 7 Collection 2 Raw Scenes. First, type ``landsat 7
collection 2'' into the search bar (Fig. F1.3.1). Without hitting Enter,
matches to that search term will appear.

\begin{figure}

{\centering \includegraphics{./F1/image67.png}

}

\caption{Fig. F1.3.1~Searching for Landsat 7 in the search bar}

\end{figure}

Now, click on~USGS Landsat 7 Collection 2 Tier 1 Raw Scenes. A new inset
window will appear (Fig. F1.3.2).

\begin{figure}

{\centering \includegraphics{./F1/image2.png}

}

\caption{Fig. F1.3.2~Inset window with information about the Landsat 7
dataset}

\end{figure}

The inset window has information about the dataset, including a
description, bands that are available, image properties, and terms of
use for the data across the top. Click on each of these tabs and read
the information provided. While you may not understand all of the
information right now, it will set you up for success in future
chapters.

On the left-hand side of this window, you will see a range of dates when
the data is available, a link to the dataset provider's webpage, and a
collection snippet. This collection snippet can be used to import the
dataset by pasting it into your script, as you did in previous chapters.
You can also use the large Import~button to import the dataset into your
current workspace. In addition, if you click on the See example~link,
Earth Engine will open a new code window with a snippet of code that
shows code using the dataset. Code snippets like this can be very
helpful when learning how to use a dataset that is new to you.

For now, click on the small ``pop out'' button in the upper right corner
of the window. This will open a new window with the same information
(Fig. F1.3.3); you can keep this new window open and use it as a
reference as you proceed.

\begin{figure}

{\centering \includegraphics{./F1/image31.png}

}

\caption{Fig. F1.3.3~The Data Catalog page for Landsat 7 with
information about the dataset}

\end{figure}

Switch back to your code window. Your ``landsat 7 collection 2'' search
term should still be in the search bar. This time, click the
``Enter''~key or click on the search magnifying glass icon. This will
open a Search results~inset window (Fig. F1.3.4).

\begin{figure}

{\centering \includegraphics{./F1/image11.png}

}

\caption{Fig. F1.3.4~Search results matching ``landsat 7 collection 2''}

\end{figure}

This more complete search results inset window contains short
descriptions about each of the datasets matching your search, to help
you choose which dataset you want to use. Click on the Open in
Catalog~button to view these search results in the Earth Engine Data
Catalog (Fig. F1.3.5). Note that you may need to click Enter~in the data
catalog search~bar with your phrase to bring up the results in this new
window.

\begin{figure}

{\centering \includegraphics{./F1/image44.png}

}

\caption{Fig. F1.3.5~Earth Engine Data Catalog results for the ``landsat
7 collection 2'' search term}

\end{figure}

Now that we know how to view this information, let's dive into some
important remote sensing terminology.

\hypertarget{spatial-resolution-1}{%
\subsection{Spatial Resolution}\label{spatial-resolution-1}}

Spatial~resolution~relates to the amount of Earth's surface area covered
by a single pixel. For example, we typically say that Landsat~7 has ``30
m'' color imagery. This means that each pixel is 30 m to a side,
covering a total area of 900 square meters~of the Earth's surface. The
spatial resolution of a given data set greatly affects the appearance of
images, and the information in them, when you are viewing them on
Earth's surface.

Next, we will visualize data from multiple sensors that capture data at
different spatial resolutions, to compare the effect of different pixel
sizes on the information and detail in an image. We'll be selecting a
single image from each ImageCollection~to visualize. To view the image,
we will draw them each as a color-IR image, a type of false-color image
(described in detail in~Chap. F1.1) that uses the infrared, red, and
green bands. As you move through this portion of the course, zoom in and
out to see differences in the pixel size and the image size.

\hypertarget{landsat-thematic-mapper}{%
\subsubsection{Landsat Thematic Mapper}\label{landsat-thematic-mapper}}

Thematic Mapper (TM) sensors were flown aboard Landsat 4 and 5. TM data
have been processed to a spatial resolution of 30m, and were active from
1982 to 2012. Search for ``Landsat 5 TM'' and import the result called
``USGS Landsat 5 TM Collection 2 Tier 1 Raw Scenes''. In this dataset,
the three bands for a color-IR image are called ``B4'' (infrared),
``B3'' (red), and ``B2'' (green). Let's now visualize TM data over San
Francisco airport. Note that we can either define the visualization
parameters as a variable (as in the previous code snippet) or place them
in curly~braces in the Map.addLayer~function (as in this code snippet).

When you run this code, the TM image will display. Notice how many more
pixels are displayed on your screen when compared to the MODIS image.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// TM  }
\CommentTok{// Filter TM imagery by location and date.  }
\NormalTok{var tmImage }\OperatorTok{=}\NormalTok{ tm  }\AttributeTok{ }\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}1987{-}03{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1987{-}08{-}01\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the TM image as a false color composite.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(tmImage}\OperatorTok{,}\NormalTok{ \{  }
  \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
  \DataTypeTok{ max}\OperatorTok{:} \DecValTok{100}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}TM\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F1/image20.png}

}

\caption{Fig. F1.3.10~Visualizing the TM imagery from the Landsat 5
satellite}

\end{figure}

\hypertarget{sentinel-2-multispectral-instrument}{%
\subsubsection{Sentinel-2 MultiSpectral
Instrument}\label{sentinel-2-multispectral-instrument}}

The MultiSpectral Instrument (MSI) flies aboard the Sentinel-2
satellites, which are operated by the European Space Agency. The red,
green, blue, and near-infrared bands are captured at 10m resolution,
while other bands are captured at 20m and 30m. The Sentinel-2A satellite
was launched in 2015 and the 2B satellite was launched in 2017.

Search for ``Sentinel 2 MSI'' in the search bar, and add the
``Sentinel-2 MSI: MultiSpectral Instrument, Level-1C'' dataset to your
workspace. Name it msi. In this dataset, the three bands for a color-IR
image are called ``B8'' (infrared), ``B4'' (red), and ``B3'' (green).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// MSI  }
\CommentTok{// Filter MSI imagery by location and date.  }
\NormalTok{var msiImage }\OperatorTok{=}\NormalTok{ msi  }\AttributeTok{ }\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}02{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2020{-}04{-}01\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the MSI image as a false color composite.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(msiImage}\OperatorTok{,}\NormalTok{ \{  }
  \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
  \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
  \DataTypeTok{ max}\OperatorTok{:} \DecValTok{2000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}MSI\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Compare the Sentinel imagery with the Landsat imagery, using the
opacity~slider. Notice how much more detail you can see on the airport
terminal and surrounding landscape. The 10 m spatial resolution means
that each pixel covers approximately 100 m2~of the Earth's surface, a
much smaller area than the TM~imagery (900 m2).

\begin{figure}

{\centering \includegraphics{./F1/image1.png}

}

\caption{Fig. F1.3.11~Visualizing the MSI imagery}

\end{figure}

\hypertarget{national-agriculture-imagery-program-naip}{%
\subsubsection{National Agriculture Imagery Program
(NAIP)}\label{national-agriculture-imagery-program-naip}}

The National Agriculture Imagery Program (NAIP) is a U.S. government
program to acquire imagery over the continental United States using
airborne sensors. Data is collected for each state approximately every
three years. The imagery has a spatial resolution of 0.5--2 m, depending
on the state and the date collected. ~

Search for ``naip'' and import the data set for ``NAIP: National
Agriculture Imagery Program''. ~Name the import naip. In this dataset,
the three bands for a color-IR image are called ``N'' (infrared), ``R''
(red), and ``G'' (green).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// NAIP  }
\CommentTok{// Get NAIP images for the study period and region of interest.  }
\NormalTok{var naipImage }\OperatorTok{=}\NormalTok{ naip}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2018{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2018{-}12{-}31\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the NAIP mosaic as a color{-}IR composite.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(naipImage}\OperatorTok{,}\NormalTok{ \{  }
  \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}N\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}R\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}G\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NAIP\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The NAIP imagery is even~more spatially detailed than the Sentinel-2 MSI
imagery. However, we can see that our one NAIP image doesn't totally
cover the San Francisco airport. If you like, zoom out to see the
boundaries of the NAIP image as we did for the Sentinel-2 MSI imagery.

\begin{figure}

{\centering \includegraphics{./F1/image32.png}

}

\caption{Fig. F1.3.13~NAIP color-IR composite over the San Francisco
airport}

\end{figure}

Each of the datasets we've examined has a different spatial resolution.
By comparing the different images over the same location in space, you
have seen the differences between the large pixels of Landsat 5, the
medium pixels of Sentinel-2, and the small pixels of the NAIP. Datasets
with large-sized pixels are also called ``coarse resolution,'' those
with medium-sized pixels are also called ``moderate resolution,'' and
those with small-sized pixels are also called ``fine resolution.''

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F13a.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{temporal-resolution-1}{%
\subsection{Temporal Resolution}\label{temporal-resolution-1}}

Temporal resolution~refers to the revisit time, or temporal cadence~of a
particular sensor's image stream. Revisit time is the number of days
between sequential visits of the satellite to the same location on the
Earth's surface. Think of this as the frequency of pixels in a time
series at a given location.

\hypertarget{landsat}{%
\subsubsection{Landsat}\label{landsat}}

The Landsat satellites 5 and later are able to image a given location
every 16 days. Let's use our existing tm~dataset from Landsat 5. To see
the time series of images at a location, you can filter an
ImageCollection~to an area and date range of interest~and then print~it.
For example, to see the Landsat 5 images for three months in 1987, run
the following code:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Explore Temporal Resolution  }
\CommentTok{/////  }
\CommentTok{// Use Print to see Landsat revisit time  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Landsat{-}5 series:\textquotesingle{}}\OperatorTok{,}\NormalTok{ tm  }\AttributeTok{ }\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}1987{-}06{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1987{-}09{-}01\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Create a chart to see Landsat 5\textquotesingle{}s 16 day revisit time.  }
\NormalTok{var tmChart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(\{  }
  \DataTypeTok{ imageCollection}\OperatorTok{:}\NormalTok{ tm}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}1987{-}06{-}01\textquotesingle{}}\OperatorTok{,}       \StringTok{\textquotesingle{}1987{-}09{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
  \DataTypeTok{ region}\OperatorTok{:}\NormalTok{ sfoPoint  }
\NormalTok{\})}\OperatorTok{.}\FunctionTok{setSeriesNames}\NormalTok{([}\StringTok{\textquotesingle{}NIR\textquotesingle{}}\NormalTok{])}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Expand the features property of the printed ImageCollection~in the
Console output to see a List~of all the images in the collection.
Observe that the date of each image is part of the filename (e.g.,
LANDSAT/LT05/C02/T1/LT05\_044034\_19870628).

\begin{figure}

{\centering \includegraphics{./F1/image3.png}

}

\caption{Fig. F1.3.14~Landsat image name and feature properties}

\end{figure}

However, viewing this list doesn't make it easy to see the temporal
resolution of the dataset. We can use Earth Engine's plotting
functionality to visualize the temporal resolution of different
datasets. For each of the different temporal resolutions, we will create
a per-pixel chart of the NIR band that we mapped previously. To do this,
we will use the ui.Chart.image.series~function.

The ui.Chart.image.series~function requires you to specify a few things
in order to calculate the point to chart for each time step. First, we
filter the ImageCollection~(you can also do this outside the function
and then specify the ImageCollection~directly). We select the B4 (near
infrared) band and then select three months by using filterDate~on the
ImageCollection. Next, we need to specify the location to chart; this is
the region argument. We'll use the sfoPoint~variable we defined earlier.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a chart to see Landsat 5\textquotesingle{}s 16 day revisit time.  }
\NormalTok{var tmChart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(\{  }
  \DataTypeTok{ imageCollection}\OperatorTok{:}\NormalTok{ tm}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}1987{-}06{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1987{-}09{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
  \DataTypeTok{ region}\OperatorTok{:}\NormalTok{ sfoPoint  }
\NormalTok{\})}\OperatorTok{.}\FunctionTok{setSeriesNames}\NormalTok{([}\StringTok{\textquotesingle{}NIR\textquotesingle{}}\NormalTok{])}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

By default, this function creates a trend line. It's difficult to see
precisely when each image was collected, so let's create a specialized
chart style that adds points for each observation.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define a chart style that will let us see the individual dates.  }
\NormalTok{var chartStyle }\OperatorTok{=}\NormalTok{ \{  }
  \DataTypeTok{ hAxis}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}Date\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ vAxis}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}NIR Mean\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ series}\OperatorTok{:}\NormalTok{ \{      }\DataTypeTok{ 0}\OperatorTok{:}\NormalTok{ \{  }
          \DataTypeTok{ lineWidth}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
          \DataTypeTok{ pointSize}\OperatorTok{:} \DecValTok{6}\ErrorTok{ }\NormalTok{      \}  }
\NormalTok{   \}}\OperatorTok{,}  
\NormalTok{\}}\OperatorTok{;}\CommentTok{// Apply custom style properties to the chart.  }
\NormalTok{tmChart}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(chartStyle)}\OperatorTok{;}  
  
\CommentTok{// Print the chart.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}TM Chart\textquotesingle{}}\OperatorTok{,}\NormalTok{ tmChart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

When you print the chart, it will have a point each time an image was
collected by the TM instrument (Fig. F1.3.15). In the Console, you can
move the mouse over the different points and see more information. Also
note that you can expand the chart using the button in the upper
right-hand corner. We will see many more examples of charts,
particularly in the chapters in Part F4.

\begin{figure}

{\centering \includegraphics{./F1/image47.png}

}

\caption{Fig. F1.3.15~A chart showing the temporal cadence, or temporal
resolution of the Landsat 5 TM instrument at the San Francisco airport}

\end{figure}

\hypertarget{sentinel-2}{%
\subsubsection{Sentinel-2}\label{sentinel-2}}

The Sentinel-2 program's two satellites are in coordinated orbits, so
that each spot on Earth gets visited about every 5 days. Within Earth
Engine, images from these two sensors are pooled in the same dataset.
Let's create a chart using the MSI instrument dataset we have already
imported.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Sentinel{-}2 has a 5 day revisit time.  }
\NormalTok{var msiChart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(\{  }
  \DataTypeTok{ imageCollection}\OperatorTok{:}\NormalTok{ msi}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}B8\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}06{-}01\textquotesingle{}}\OperatorTok{,}       \StringTok{\textquotesingle{}2020{-}09{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
  \DataTypeTok{ region}\OperatorTok{:}\NormalTok{ sfoPoint  }
\NormalTok{\})}\OperatorTok{.}\FunctionTok{setSeriesNames}\NormalTok{([}\StringTok{\textquotesingle{}NIR\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Apply the previously defined custom style properties to the chart.  }
\NormalTok{msiChart}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(chartStyle)}\OperatorTok{;}  
  
\CommentTok{// Print the chart.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}MSI Chart\textquotesingle{}}\OperatorTok{,}\NormalTok{ msiChart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F1/image60.png}

}

\caption{Fig. F1.3.16~A chart showing the t temporal resolution of the
Sentinel-2 MSI instrument at the San Francisco airport}

\end{figure}

Compare this Sentinel-2 graph (Fig. F1.3.16) with the Landsat graph you
just produced (Fig. F1.3.15). Both cover a period of six months, yet
there are many more points through time for the Sentinel-2 satellite,
reflecting the greater temporal resolution.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F13b.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{spectral-resolution-1}{%
\subsection{Spectral Resolution}\label{spectral-resolution-1}}

Spectral resolution~refers to the number and width of spectral bands in
which the sensor takes measurements. You can think of the width of
spectral bands as the wavelength intervals for each band. A sensor that
measures radiance in multiple bands is called a
multispectral~sensor~(generally 3--10 bands), while a sensor with many
bands (possibly hundreds) is called a hyperspectral~sensor.

Let's compare the multispectral MODIS instrument with the hyperspectral
Hyperion~sensor aboard the EO-1 satellite, which is also available in
Earth Engine.

\hypertarget{modis}{%
\subsubsection{MODIS}\label{modis}}

There is an easy way to check the number of bands in an image:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Explore spectral resolution  }
\CommentTok{/////  }
  
\CommentTok{// Get the MODIS band names as an ee.List  }
\NormalTok{var modisBands }\OperatorTok{=}\NormalTok{ modisImage}\OperatorTok{.}\FunctionTok{bandNames}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Print the list.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}MODIS bands:\textquotesingle{}}\OperatorTok{,}\NormalTok{ modisBands)}\OperatorTok{;}  
  
\CommentTok{// Print the length of the list.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Length of the bands list:\textquotesingle{}}\OperatorTok{,}\NormalTok{ modisBands}\OperatorTok{.}\FunctionTok{length}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that not all of the bands are spectral bands. As we did with the
temporal resolution, let's graph the spectral bands to examine the
spectral resolution. If you ever have questions about what the different
bands in the band list are, remember that you can find this information
by visiting the dataset information page~in Earth Engine or the data or
satellite's webpage.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Graph the MODIS spectral bands (bands 11{-}17).  }
  
\CommentTok{// Select only the reflectance bands of interest.  }
\NormalTok{var reflectanceImage }\OperatorTok{=}\NormalTok{ modisImage}\OperatorTok{.}\FunctionTok{select}\NormalTok{(   }\StringTok{\textquotesingle{}sur\_refl\_b01\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}sur\_refl\_b02\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}sur\_refl\_b03\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}sur\_refl\_b04\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}sur\_refl\_b05\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}sur\_refl\_b06\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}sur\_refl\_b07\textquotesingle{}}  
\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

As before, we'll customize the chart to make it easier to read.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define an object of customization parameters for the chart.  }
\NormalTok{var options }\OperatorTok{=}\NormalTok{ \{  }
  \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}MODIS spectrum at SFO\textquotesingle{}}\OperatorTok{,}  
  \DataTypeTok{ hAxis}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}Band\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ vAxis}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}Reflectance\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ legend}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ position}\OperatorTok{:} \StringTok{\textquotesingle{}none\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ pointSize}\OperatorTok{:} \DecValTok{3}  
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

And create a chart using the ui.Chart.image.regions~function.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Make the chart.  }
\NormalTok{var modisReflectanceChart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{regions}\NormalTok{(\{  }
  \DataTypeTok{ image}\OperatorTok{:}\NormalTok{ reflectanceImage}\OperatorTok{,}  
  \DataTypeTok{ regions}\OperatorTok{:}\NormalTok{ sfoPoint  }
\NormalTok{\})}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(options)}\OperatorTok{;}  
  
\CommentTok{// Display the chart.  }
\FunctionTok{print}\NormalTok{(modisReflectanceChart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The resulting chart is shown in Fig. F1.3.17.~Use the expand button in
the upper right to see a larger version of the chart than the one
printed to the Console.

\begin{figure}

{\centering \includegraphics{./F1/image50.png}

}

\caption{Fig. F1.3.17~Plot of TOA reflectance~for MODIS}

\end{figure}

\hypertarget{eo-1}{%
\subsubsection{EO-1}\label{eo-1}}

Now let's compare MODIS with the EO-1 satellite's hyperspectral sensor.
Search for ``eo-1'' and import the ``EO-1 Hyperion Hyperspectral
Imager'' dataset. Name it eo1. We can look at the number of bands from
the EO-1 sensor.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Get the EO{-}1 band names as a ee.List  }
\NormalTok{var eo1Image }\OperatorTok{=}\NormalTok{ eo1  }\AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2015{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2016{-}01{-}01\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Extract the EO{-}1 band names.  }
\NormalTok{var eo1Bands }\OperatorTok{=}\NormalTok{ eo1Image}\OperatorTok{.}\FunctionTok{bandNames}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Print the list of band names.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}EO{-}1 bands:\textquotesingle{}}\OperatorTok{,}\NormalTok{ eo1Bands)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Examine the list of bands that are printed in the Console. Notice how
many more bands the hyperspectral instrument provides.

Now let's create a reflectance chart as we did with the MODIS data.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create an options object for our chart.  }
\NormalTok{var optionsEO1 }\OperatorTok{=}\NormalTok{ \{  }
  \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}EO1 spectrum\textquotesingle{}}\OperatorTok{,}  
  \DataTypeTok{ hAxis}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}Band\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ vAxis}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ title}\OperatorTok{:} \StringTok{\textquotesingle{}Reflectance\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ legend}\OperatorTok{:}\NormalTok{ \{  }
      \DataTypeTok{ position}\OperatorTok{:} \StringTok{\textquotesingle{}none\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
  \DataTypeTok{ pointSize}\OperatorTok{:} \DecValTok{3}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Make the chart and set the options.  }
\NormalTok{var eo1Chart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{regions}\NormalTok{(\{  }
  \DataTypeTok{ image}\OperatorTok{:}\NormalTok{ eo1Image}\OperatorTok{,}  
  \DataTypeTok{ regions}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\FloatTok{6.10}\OperatorTok{,} \FloatTok{81.12}\NormalTok{])  }
\NormalTok{\})}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(optionsEO1)}\OperatorTok{;}  
  
\CommentTok{// Display the chart.  }
\FunctionTok{print}\NormalTok{(eo1Chart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The resulting chart is seen in Fig. F1.3.18. There are so many bands
that their names only appear as ``\ldots{}''!

\begin{figure}

{\centering \includegraphics{./F1/image23.png}

}

\caption{Fig. F1.3.18~Plot of TOA reflectance~for EO-1 as displayed in
the Console. Note the button to expand the plot in the upper right hand
corner.}

\end{figure}

If we click on the expand icon in the top right corner of the chart,
it's a little easier to see the band identifiers, as shown in Fig.
F1.3.19.~

\begin{figure}

{\centering \includegraphics{./F1/image70.png}

}

\caption{Fig. F1.3.19~Expanded plot of TOA reflectance~for EO-1}

\end{figure}

Compare this hyperspectral instrument chart with the multispectral chart
we plotted above for MODIS.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F13c.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{per-pixel-quality}{%
\subsection{Per-Pixel Quality}\label{per-pixel-quality}}

As you saw above, an image consists of many bands. Some of these bands
contain spectral responses of Earth's surface, including the NIR, red,
and green bands we examined in the Spectral Resolution section. What
about the other bands? Some of these other bands contain valuable
information, like pixel-by-pixel quality-control data.

For example, Sentinel-2 has a QA60 band, which contains the surface
reflectance quality assurance~information. Let's map it to inspect the
values.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Examine pixel quality  }
\CommentTok{/////  }
  
\CommentTok{// Sentinel Quality Visualization.  }
\NormalTok{var msiCloud }\OperatorTok{=}\NormalTok{ msi  }\AttributeTok{ }\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2019{-}12{-}31\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2020{-}02{-}01\textquotesingle{}}\NormalTok{)  }
  \AttributeTok{ }\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the MSI image as a false color composite.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(msiCloud}\OperatorTok{,}  
\NormalTok{   \{  }
      \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{2000}\ErrorTok{ }\NormalTok{  \}}\OperatorTok{,}   \StringTok{\textquotesingle{}MSI Quality Image\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(msiCloud}\OperatorTok{,}  
\NormalTok{   \{  }
      \DataTypeTok{ bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}QA60\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
      \DataTypeTok{ min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
      \DataTypeTok{ max}\OperatorTok{:} \DecValTok{2000}\ErrorTok{ }\NormalTok{  \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Sentinel Quality Visualization\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Use the Inspector~tool to examine some of the values. You may see values
of 0 (black), 1024 (gray), and 2048 (white). The QA60 band has values of
1024~for opaque clouds, and 2048 for cirrus clouds. Compare the
false-color image with the QA60 band to see these values. More
information about how to interpret these complex values is given in
Chap. F4.3,~which explains the treatment of clouds.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F13d.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{metadata}{%
\subsection{Metadata}\label{metadata}}

In addition to band imagery and per-pixel quality flags, Earth Engine
allows you to access substantial amounts of metadata associated with an
image. This can all be easily printed to the Console~for a single image.

Let's examine the metadata for the Sentinel-2 MSI.

/////\\
// Metadata\\
/////\\
print(`MSI Image Metadata', msiImage);

Examine the object you've created in the Console (Fig. F1.3.20). Expand
the image name, then the properties~object.

\includegraphics{./F1/image35.png}

Fig. F1.3.20 Checking the ``CLOUDY\_PIXEL\_PERCENTAGE'' property in the
metadata for Sentinel-2

The first entry is the CLOUDY\_PIXEL\_PERCENTAGE~information. Distinct
from the cloudiness flag attached to every pixel, this is an image-level
summary assessment of the overall cloudiness in the image. In addition
to viewing the value, you might find it useful to~print it to the
screen, for example, or to record a list of cloudiness values in a set
of images. Metadata properties can~be extracted from an image's
properties using the get~function, and printed to the Console.

// Image-level Cloud info\\
var msiCloudiness = msiImage.get(`CLOUDY\_PIXEL\_PERCENTAGE');

print(`MSI CLOUDY\_PIXEL\_PERCENTAGE:', msiCloudiness);

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F13e.~The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-3}{%
\subsection*{Conclusion}\label{conclusion-3}}
\addcontentsline{toc}{subsection}{Conclusion}

A good understanding of the characteristics of your images~is critical
to your work in Earth Engine and the chapters going forward. You now
know how to observe and query a variety of remote sensing~datasets, and
can choose among them for your work. For~example, if you are interested
in change detection, you might require a dataset with spectral
resolution including near-infrared imagery and a fine temporal
resolution. For analyses at a continental scale, you may prefer data
with a coarse spatial scale, while analyses for specific forest stands
may benefit from a very fine spatial scale.

\hypertarget{references-1}{%
\subsection*{References}\label{references-1}}
\addcontentsline{toc}{subsection}{References}

Fisher JRB, Acosta EA, Dennedy-Frank PJ, et al (2018) Impact of
satellite imagery spatial resolution on land use classification accuracy
and modeled water quality. Remote Sens Ecol Conserv 4:137--149.
https://doi.org/10.1002/rse2.61

\hypertarget{interpreting-images}{%
\chapter{Interpreting Images}\label{interpreting-images}}

Now that you know how images are viewed and what kinds of images exist
in Earth Engine, how do we manipulate them? To gain the skills of
interpreting images, you'll work with bands, combining values to form
indices and masking unwanted pixels. Then, you'll learn some of the
techniques available in Earth Engine for classifying images and
interpreting the results.

\hypertarget{image-manipulation-bands-arithmetic-thresholds-and-masks}{%
\section{Image Manipulation: Bands, Arithmetic, Thresholds, and
Masks}\label{image-manipulation-bands-arithmetic-thresholds-and-masks}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-2}{%
\subsubsection*{Author}\label{author-2}}

Karen Dyson, Andréa Puzzi Nicolau, David Saah, and Nicholas Clinton

\hypertarget{overview-4}{%
\subsubsection*{Overview}\label{overview-4}}

Once images have been identified in Earth Engine, they can be viewed in
a wide array of band combinations for targeted purposes. For users who
are already versed in remote sensing concepts, this chapter shows how to
do familiar tasks on this platform; for those who are entirely new to
such concepts, it introduces the idea of band combinations.

\hypertarget{learning-outcomes-4}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-4}}

\begin{itemize}
\tightlist
\item
  Understanding what spectral indices are and why they are useful.
\item
  Being introduced to a range of example spectral indices used for a
  variety of purposes.
\end{itemize}

\hypertarget{assumes-you-know-how-to-4}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-4}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-3}{%
\subsection*{Introduction}\label{introduction-3}}

Spectral indices are based on the fact that different objects and land
covers on the Earth's surface reflect different amounts of light from
the Sun at different wavelengths. In the visible part of the spectrum,
for example, a healthy green plant reflects a large amount of green
light while absorbing blue and red light---which is why it appears green
to our eyes. Light also arrives from the Sun at wavelengths outside what
the human eye can see, and there are large differences in reflectances
between living and nonliving land covers, and between different types of
vegetation, both in the visible and outside the visible wavelengths. We
visualized this earlier, in Chaps. F1.1 and F1.3 when we mapped
color-infrared images (Fig. F2.0.1).

\begin{figure}

{\centering \includegraphics{./F2/image39.png}

}

\caption{Fig. F2.0.1 Mapped color-IR images from multiple satellite
sensors that we mapped in Chap. F1.3. The near infrared spectrum is
mapped as red, showing where there are high amounts of healthy
vegetation.}

\end{figure}

If we graph the amount of light (reflectance) at different wavelengths
that an object or land cover reflects, we can visualize this more easily
(Fig. F2.0.2). For example, look at the reflectance curves for soil and
water in the graph below. Soil and water both have relatively low
reflectance at wavelengths around 300 nm (ultraviolet and violet light).
Conversely, at wavelengths above 700 nm (red and infrared light) soil
has relatively high reflectance, while water has very low reflectance.
Vegetation, meanwhile, generally reflects large amounts of near infrared
light, relative to other land covers.

\begin{figure}

{\centering \includegraphics{./F2/image32.png}

}

\caption{Fig. F2.0.2 A graph of the amount of reflectance for different
objects on the Earth's surface at different wavelengths in the visible
and infrared portions of the electromagnetic spectrum. 1 micrometer (µm)
= 1,000 nanometers (nm).}

\end{figure}

Spectral indices use math to express how objects reflect light across
multiple portions of the spectrum as a single number. Indices combine
multiple bands, often with simple operations of subtraction and
division, to create a single value across an image that is intended to
help to distinguish particular land uses or land covers of interest.
Using Fig. F2.0.2, you can imagine which wavelengths might be the most
informative for distinguishing among a variety of land covers. We will
explore a variety of calculations made from combinations of bands in the
following sections.

Indices derived from satellite imagery are used as the basis of many
remote-sensing analyses. Indices have been used in thousands of
applications, from detecting anthropogenic deforestation to examining
crop health. For example, the growth of economically important crops
such as wheat and cotton can be monitored throughout the growing season:
Bare soil reflects more red wavelengths, whereas growing crops reflect
more of the near-infrared (NIR) wavelengths. Thus, calculating a ratio
of these two bands can help monitor how well crops are growing (Jackson
and Huete 1991).

\hypertarget{band-arithmetic-in-earth-engine}{%
\subsection{Band Arithmetic in Earth
Engine}\label{band-arithmetic-in-earth-engine}}

If you have not already done so, be sure to add the book's code
repository to the Code Editor by entering
\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458829783542\&usg=AOvVaw2f8xfEZP6c0zP_Ke8jL26U}{}\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458829783919\&usg=AOvVaw2i09J44MzpMZkjV_JLEnNR}{https://code.earthengine.google.com/?accept\_repo=projects/gee-edu/book}
into your browser. The book's scripts will then be available in the
script manager panel. If you have trouble finding the repo, you can
visit
\href{https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit\%23slide\%3Did.g18a7b4b055d_0_624\&sa=D\&source=editors\&ust=1671458829784270\&usg=AOvVaw1Kr82KG60ZeFLYC8cOZ67A}{this
link} for help.

Many indices can be calculated using band arithmetic in Earth Engine.
Band arithmetic is the process of adding, subtracting, multiplying, or
dividing two or more bands from an image. Here we'll first do this
manually, and then show you some more efficient ways to perform band
arithmetic in Earth Engine.

\hypertarget{arithmetic-calculation-of-ndvi}{%
\subsubsection{Arithmetic Calculation of
NDVI}\label{arithmetic-calculation-of-ndvi}}

The red and near-infrared bands provide a lot of information about
vegetation due to vegetation's high reflectance in these wavelengths.
Take a look at Fig. F2.0.2 and note, in particular, that vegetation
curves (graphed in green) have relatively high reflectance in the NIR
range (approximately 750--900 nm). Also note that vegetation has low
reflectance in the red range (approximately 630--690 nm), where sunlight
is absorbed by chlorophyll. This suggests that if the red and
near-infrared bands could be combined, they would provide substantial
information about vegetation.

Soon after the launch of Landsat 1 in 1972, analysts worked to devise a
robust single value that would convey the health of vegetation along a
scale of −1 to 1. This yielded the NDVI, using the formula:

\includegraphics{./F2/image1.png} (F2.0.1)

where NIR and red refer to the brightness of each of those two bands. As
seen in Chaps. F1.1 and F1.2, this brightness might be conveyed in units
of reflectance, radiance, or digital number (DN); the NDVI is intended
to give nearly equivalent values across platforms that use these
wavelengths. The general form of this equation is called a ``normalized
difference''---the numerator is the ``difference'' and the denominator
``normalizes'' the value. Outputs for NDVI vary between −1 and 1. High
amounts of green vegetation have values around 0.8--0.9. Absence of
green leaves gives values near 0, and water gives values near −1.

To compute the NDVI, we will introduce Earth Engine's implementation of
band arithmetic. Cloud-based band arithmetic is one of the most powerful
aspects of Earth Engine, because the platform's computers are optimized
for this type of heavy processing. Arithmetic on bands can be done even
at planetary scale very quickly---an idea that was out of reach before
the advent of cloud-based remote sensing. Earth Engine automatically
partitions calculations across a large number of computers as needed,
and assembles the answer for display.

As an example, let's examine an image of San Francisco
(\hspace{0pt}\hspace{0pt}Fig. F2.0.3).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////  }
\CommentTok{// Band Arithmetic  }
\CommentTok{/////  }
  
\CommentTok{// Calculate NDVI using Sentinel 2  }
  
\CommentTok{// Import and filter imagery by location and date.  }
\KeywordTok{var}\NormalTok{ sfoPoint }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{(}\OperatorTok{{-}}\FloatTok{122.3774}\OperatorTok{,} \FloatTok{37.6194}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ sfoImage }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(sfoPoint)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}02{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2020{-}04{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the image as a false color composite.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(sfoImage}\OperatorTok{,} \DecValTok{11}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfoImage}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{2000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}False color\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image46.png}

}

\caption{Fig. F2.0.3 False color Sentinel-2 imagery of San Francisco and
surroundings}

\end{figure}

The simplest mathematical operations in Earth Engine are the add,
subtract, multiply, and divide methods. Let's select the near-infrared
and red bands and use these operations to calculate NDVI for our image.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Extract the near infrared and red bands.  }
\KeywordTok{var}\NormalTok{ nir }\OperatorTok{=}\NormalTok{ sfoImage}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}B8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ red }\OperatorTok{=}\NormalTok{ sfoImage}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Calculate the numerator and the denominator using subtraction and addition respectively.  }
\KeywordTok{var}\NormalTok{ numerator }\OperatorTok{=}\NormalTok{ nir}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(red)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ denominator }\OperatorTok{=}\NormalTok{ nir}\OperatorTok{.}\FunctionTok{add}\NormalTok{(red)}\OperatorTok{;}  
  
\CommentTok{// Now calculate NDVI.  }
\KeywordTok{var}\NormalTok{ ndvi }\OperatorTok{=}\NormalTok{ numerator}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(denominator)}\OperatorTok{;}  
  
\CommentTok{// Add the layer to our map with a palette.  }
\KeywordTok{var}\NormalTok{ vegPalette }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(ndvi}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ vegPalette  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI Manual\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Examine the resulting index, using the Inspector to pick out pixel
values in areas of vegetation and non-vegetation if desired.

\begin{figure}

{\centering \includegraphics{./F2/image50.png}

}

\caption{Fig. F2.0.4 NDVI calculated using Sentinel-2. Remember that
outputs for NDVI vary between −1 and 1. High amounts of green vegetation
have values around 0.8--0.9. Absence of green leaves gives values near
0, and water gives values near −1.}

\end{figure}

Using these simple arithmetic tools, you can build almost any index, or
develop and visualize your own. Earth Engine allows you to quickly and
easily calculate and display the index across a large area.

\hypertarget{single-operation-computation-of-normalized-difference-for-ndvi}{%
\subsubsection{Single-Operation Computation of Normalized Difference for
NDVI}\label{single-operation-computation-of-normalized-difference-for-ndvi}}

Normalized differences like NDVI are so common in remote sensing that
Earth Engine provides the ability to do that particular sequence of
subtraction, addition, and division in a single step, using the
normalizedDifference method. This method takes an input image, along
with bands you specify, and creates a normalized difference of those two
bands. The NDVI computation previously created with band arithmetic can
be replaced with one line of code:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Now use the built{-}in normalizedDifference function to achieve the same outcome.  }
\KeywordTok{var}\NormalTok{ ndviND }\OperatorTok{=}\NormalTok{ sfoImage}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(ndviND}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ vegPalette  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI normalizedDiff\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that the order in which you provide the two bands to
normalizedDifference is important. We use B8, the near-infrared band, as
the first parameter, and the red band B4 as the second. If your two
computations of NDVI do not look identical when drawn to the screen,
check to make sure that the order you have for the NIR and red bands is
correct.

\hypertarget{using-normalized-difference-for-ndwi}{%
\subsubsection{Using Normalized Difference for
NDWI}\label{using-normalized-difference-for-ndwi}}

As mentioned, the normalized difference approach is used for many
different indices. Let's apply the same normalizedDifference method to
another index.

The Normalized Difference Water Index (NDWI) was developed by Gao (1996)
as an index of vegetation water content. The index is sensitive to
changes in the liquid content of vegetation canopies. This means that
the index can be used, for example, to detect vegetation experiencing
drought conditions or differentiate crop irrigation levels. In dry
areas, crops that are irrigated can be differentiated from natural
vegetation. It is also sometimes called the Normalized Difference
Moisture Index (NDMI). NDWI is formulated as follows:

\begin{figure}

{\centering \includegraphics{./F2/image2.png}

}

\caption{(F2.0.2)}

\end{figure}

where NIR is near-infrared, centered near 860 nm (0.86 μm), and SWIR is
short-wave infrared, centered near 1,240 nm (1.24 μm).

Compute and display NDWI in Earth Engine using the normalizedDifference
method. Remember that for Sentinel-2, B8 is the NIR band and B11 is the
SWIR band (refer to Chaps. F1.1 and F1.3 to find information about
imagery bands).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Use normalizedDifference to calculate NDWI  }
\KeywordTok{var}\NormalTok{ ndwi }\OperatorTok{=}\NormalTok{ sfoImage}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B11\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ waterPalette }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}blue\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(ndwi}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.5}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ waterPalette  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NDWI\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Examine the areas of the map that NDVI identified as having a lot of
vegetation. Notice which are more blue. This is vegetation that has
higher water content.

\begin{figure}

{\centering \includegraphics{./F2/image40.png}

}

\caption{Fig. F2.0.5 NDWI displayed for Sentinel-2 over San Francisco}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F20a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{thresholding-masking-and-remapping-images}{%
\subsection{Thresholding, Masking, and Remapping
Images}\label{thresholding-masking-and-remapping-images}}

The previous section in this chapter discussed how to use band
arithmetic to manipulate images. Those methods created new continuous
values by combining bands within an image. This section uses logical
operators to categorize band or index values to create a categorized
image.

\hypertarget{implementing-a-threshold}{%
\subsubsection{Implementing a
Threshold}\label{implementing-a-threshold}}

Implementing a threshold uses a number (the threshold value) and logical
operators to help us partition the variability of images into
categories. For example, recall our map of NDVI. High amounts of
vegetation have NDVI values near 1 and non-vegetated areas are near 0.
If we want to see what areas of the map have vegetation, we can use a
threshold to generalize the NDVI value in each pixel as being either
``no vegetation'' or ``vegetation''. That is a substantial
simplification, to be sure, but can help us to better comprehend the
rich variation on the Earth's surface. This type of categorization may
be useful if, for example, we want to look at the proportion of a city
that is vegetated. Let's create a Sentinel-2 map of NDVI near Seattle,
Washington, USA. Enter the code below in a new script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create an NDVI image using Sentinel 2.  }
\KeywordTok{var}\NormalTok{ seaPoint }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{(}\OperatorTok{{-}}\FloatTok{122.2040}\OperatorTok{,} \FloatTok{47.6221}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ seaImage }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(seaPoint)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}08{-}15\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2020{-}10{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ seaNDVI }\OperatorTok{=}\NormalTok{ seaImage}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// And map it.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(seaPoint}\OperatorTok{,} \DecValTok{10}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ vegPalette }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(seaNDVI}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ vegPalette  }
\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}NDVI Seattle\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image30.png}

}

\caption{Fig. F2.0.6 NDVI image of Sentinel-2 imagery over Seattle,
Washington, USA}

\end{figure}

Inspect the image. We can see that vegetated areas are darker green
while non-vegetated locations are white and water is pink. If we use the
Inspector to query our image, we can see that parks and other forested
areas have an NDVI over about 0.5. Thus, it would make sense to define
areas with NDVI values greater than 0.5 as forested, and those below
that threshold as not forested.

Now let's define that value as a threshold and use it to threshold our
vegetated areas.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Implement a threshold.  }
\KeywordTok{var}\NormalTok{ seaVeg }\OperatorTok{=}\NormalTok{ seaNDVI}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\FloatTok{0.5}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Map the threshold.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(seaVeg}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]  }
\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Non{-}forest vs. Forest\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The gt method is from the family of Boolean operators---that is, gt is a
function that performs a test in each pixel and returns the value 1 if
the test evaluates to true, and 0 otherwise. Here, for every pixel in
the image, it tests whether the NDVI value is greater than 0.5. When
this condition is met, the layer seaVeg gets the value 1. When the
condition is false, it receives the value 0.

\begin{figure}

{\centering \includegraphics{./F2/image47.png}

}

\caption{Fig. F2.0.7 Thresholded forest and non-forest image based on
NDVI for Seattle, Washington, USA}

\end{figure}

Use the Inspector tool to explore this new layer. If you click on a
green location, that NDVI should be greater than 0.5. If you click on a
white pixel, the NDVI value should be equal to or less than 0.5.

Other operators in this Boolean family include less than (lt), less than
or equal to (lte), equal to (eq), not equal to (neq), and greater than
or equal to (gte) and more.

\hypertarget{building-complex-categorizations-with-.where}{%
\subsubsection{Building Complex Categorizations with
.where}\label{building-complex-categorizations-with-.where}}

A binary map classifying NDVI is very useful. However, there are
situations where you may want to split your image into more than two
bins. Earth Engine provides a tool, the where method, that conditionally
evaluates to true or false within each pixel depending on the outcome of
a test. This is analogous to an if statement seen commonly in other
languages. However, to perform this logic when programming for Earth
Engine, we avoid using the JavaScript if statement. Importantly,
JavaScript if commands are not calculated on Google's servers, and can
create serious problems when running your code---in effect, the servers
try to ship all of the information to be executed to your own computer's
browser, which is very underequipped for such enormous tasks. Instead,
we use the where clause for conditional logic.

Suppose instead of just splitting the forested areas from the
non-forested areas in our NDVI, we want to split the image into likely
water, non-forested, and forested areas. We can use where and thresholds
of -0.1 and 0.5. We will start by creating an image using ee.Image. We
then clip the new image so that it covers the same area as our seaNDVI
layer.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Implement .where.  }
\CommentTok{// Create a starting image with all values = 1.  }
\KeywordTok{var}\NormalTok{ seaWhere }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\DecValTok{1}\NormalTok{)   }\CommentTok{// Use clip to constrain the size of the new image.   .clip(seaNDVI.geometry());  }
  
\CommentTok{// Make all NDVI values less than {-}0.1 equal 0.  }
\NormalTok{seaWhere }\OperatorTok{=}\NormalTok{ seaWhere}\OperatorTok{.}\FunctionTok{where}\NormalTok{(seaNDVI}\OperatorTok{.}\FunctionTok{lte}\NormalTok{(}\OperatorTok{{-}}\FloatTok{0.1}\NormalTok{)}\OperatorTok{,} \DecValTok{0}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Make all NDVI values greater than 0.5 equal 2.  }
\NormalTok{seaWhere }\OperatorTok{=}\NormalTok{ seaWhere}\OperatorTok{.}\FunctionTok{where}\NormalTok{(seaNDVI}\OperatorTok{.}\FunctionTok{gte}\NormalTok{(}\FloatTok{0.5}\NormalTok{)}\OperatorTok{,} \DecValTok{2}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Map our layer that has been divided into three classes.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(seaWhere}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{2}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]  }
\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Water, Non{-}forest, Forest\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

There are a few interesting things to note about this code that you may
not have seen before. First, we're not defining a new variable for each
where call. As a result, we can perform many where calls without
creating a new variable each time and needing to keep track of them.
Second, when we created the starting image, we set the value to 1. This
means that we could easily set the bottom and top values with one where
clause each. Finally, while we did not do it here, we can combine
multiple where clauses using and and or. For example, we could identify
pixels with an intermediate level of NDVI using
seaNDVI.gte(-0.1).and(seaNDVI.lt(0.5)).

\begin{figure}

{\centering \includegraphics{./F2/image37.png}

}

\caption{Fig. F2.0.8 Thresholded water, forest, and non-forest image
based on NDVI for Seattle, Washington, USA.}

\end{figure}

\hypertarget{masking-specific-values-in-an-image}{%
\subsubsection{Masking Specific Values in an
Image}\label{masking-specific-values-in-an-image}}

Masking an image is a technique that removes specific areas of an
image---those covered by the mask---from being displayed or analyzed.
Earth Engine allows you to both view the current mask and update the
mask.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Implement masking.  }
\CommentTok{// View the seaVeg layer\textquotesingle{}s current mask.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(seaPoint}\OperatorTok{,} \DecValTok{9}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(seaVeg}\OperatorTok{.}\FunctionTok{mask}\NormalTok{()}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}seaVeg Mask\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image23.png}

}

\caption{Fig. F2.0.9 The existing mask for the seaVeg layer we created
previously}

\end{figure}

You can use the Inspector to see that the black area is masked and the
white area has a constant value of 1. This means that data values are
mapped and available for analysis within the white area only.

Now suppose we only want to display and conduct analyses in the forested
areas. Let's mask out the non-forested areas from our image. First, we
create a binary mask using the equals (eq) method.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a binary mask of non{-}forest.  }
\KeywordTok{var}\NormalTok{ vegMask }\OperatorTok{=}\NormalTok{ seaVeg}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In making a mask, you set the values you want to see and analyze to be a
number greater than 0. The idea is to set unwanted values to get the
value of 0. Pixels that had 0 values become masked out (in practice,
they do not appear on the screen at all) once we use the updateMask
method to add these values to the existing mask.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Update the seaVeg mask with the non{-}forest mask.  }
\KeywordTok{var}\NormalTok{ maskedVeg }\OperatorTok{=}\NormalTok{ seaVeg}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(vegMask)}\OperatorTok{;}  
  
\CommentTok{// Map the updated Veg layer  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(maskedVeg}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]  }
\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Masked Forest Layer\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Turn off all of the other layers. You can see how the maskedVeg layer
now has masked out all non-forested areas.

\begin{figure}

{\centering \includegraphics{./F2/image26.png}

}

\caption{Fig. F2.0.10 An updated mask now displays only the forested
areas. Non-forested areas are masked out and transparent.}

\end{figure}

Map the updated mask for the layer and you can see why this is.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Map the updated mask  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(maskedVeg}\OperatorTok{.}\FunctionTok{mask}\NormalTok{()}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}maskedVeg Mask\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image33.png}

}

\caption{Fig. F2.0.11 The updated mask. Areas of non-forest are now
masked out as well (black areas of the image).}

\end{figure}

\hypertarget{remapping-values-in-an-image}{%
\subsubsection{Remapping Values in an
Image}\label{remapping-values-in-an-image}}

Remapping takes specific values in an image and assigns them a different
value. This is particularly useful for categorical datasets, including
those you read about in Chap. F1.2 and those we have created earlier in
this chapter.

Let's use the remap method to change the values for our seaWhere layer.
Note that since we're changing the middle value to be the largest, we'll
need to adjust our palette as well.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Implement remapping.  }
\CommentTok{// Remap the values from the seaWhere layer.  }
\KeywordTok{var}\NormalTok{ seaRemap }\OperatorTok{=}\NormalTok{ seaWhere}\OperatorTok{.}\FunctionTok{remap}\NormalTok{([}\DecValTok{0}\OperatorTok{,} \DecValTok{1}\OperatorTok{,} \DecValTok{2}\NormalTok{]}\OperatorTok{,} \CommentTok{// Existing values.   [9, 11, 10]); // Remapped values.  }
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(seaRemap}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{9}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{11}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\NormalTok{]  }
\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Remapped Values\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Use the inspector to compare values between our original seaWhere
(displayed as Water, Non-Forest, Forest) and the seaRemap, marked as
``Remapped Values.'' Click on a forested area and you should see that
the Remapped Values should be 10, instead of 2 (Fig. F2.0.12).

\begin{figure}

{\centering \includegraphics{./F2/image28.png}

}

\caption{Fig. F2.0.12 For forested areas, the remapped layer has a value
of 10, compared with the original layer, which has a value of 2. You may
have more layers in your Inspector.}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F20b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-4}{%
\subsection*{Conclusion}\label{conclusion-4}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to select multiple bands from an image
and calculate indices. You also learned about thresholding values in an
image, slicing them into multiple categories using thresholds. It is
also possible to work with one set of class numbers and remap them
quickly to another set. Using these techniques, you have some of the
basic tools of image manipulation. In subsequent chapters you will
encounter more complex and specialized image manipulation techniques,
including pixel-based image transformations (Chap. F3.1),
neighborhood-based image transformations (Chap. F3.2), and object-based
image analysis (Chap. F3.3).

\hypertarget{references-2}{%
\subsection*{References}\label{references-2}}
\addcontentsline{toc}{subsection}{References}

Baig MHA, Zhang L, Shuai T, Tong Q (2014) Derivation of a tasselled cap
transformation based on Landsat 8 at-satellite reflectance. Remote Sens
Lett 5:423--431. https://doi.org/10.1080/2150704X.2014.915434

Crist EP (1985) A TM tasseled cap equivalent transformation for
reflectance factor data. Remote Sens Environ 17:301--306.
https://doi.org/10.1016/0034-4257(85)90102-6

Drury SA (1987) Image interpretation in geology. Geocarto Int 2:48.
https://doi.org/10.1080/10106048709354098

Gao BC (1996) NDWI - A normalized difference water index for remote
sensing of vegetation liquid water from space. Remote Sens Environ
58:257--266. https://doi.org/10.1016/S0034-4257(96)00067-3

Huang C, Wylie B, Yang L, et al (2002) Derivation of a tasselled cap
transformation based on Landsat 7 at-satellite reflectance. Int J Remote
Sens 23:1741--1748. https://doi.org/10.1080/01431160110106113

Jackson RD, Huete AR (1991) Interpreting vegetation indices. Prev Vet
Med 11:185--200. https://doi.org/10.1016/S0167-5877(05)80004-2

Martín MP (1998) Cartografía e inventario de incendios forestales en la
Península Ibérica a partir de imágenes NOAA-AVHRR. Universidad de Alcalá

McFeeters SK (1996) The use of the Normalized Difference Water Index
(NDWI) in the delineation of open water features. Int J Remote Sens
17:1425--1432. https://doi.org/10.1080/01431169608948714

Nath B, Niu Z, Mitra AK (2019) Observation of short-term variations in
the clay minerals ratio after the 2015 Chile great earthquake (8.3 Mw)
using Landsat 8 OLI data. J Earth Syst Sci 128:1--21.
https://doi.org/10.1007/s12040-019-1129-2

Schultz M, Clevers JGPW, Carter S, et al (2016) Performance of
vegetation indices from Landsat time series in deforestation monitoring.
Int J Appl Earth Obs Geoinf 52:318--327.
https://doi.org/10.1016/j.jag.2016.06.020

Segal D (1982) Theoretical basis for differentiation of ferric-iron
bearing minerals, using Landsat MSS data. In: Proceedings of Symposium
for Remote Sensing of Environment, 2nd Thematic Conference on Remote
Sensing for Exploratory Geology, Fort Worth, TX. pp 949--951

Souza Jr CM, Roberts DA, Cochrane MA (2005) Combining spectral and
spatial information to map canopy damage from selective logging and
forest fires. Remote Sens Environ 98:329--343.
https://doi.org/10.1016/j.rse.2005.07.013

Souza Jr CM, Siqueira JV, Sales MH, et al (2013) Ten-year Landsat
classification of deforestation and forest degradation in the Brazilian
Amazon. Remote Sens 5:5493--5513. https://doi.org/10.3390/rs5115493

\hypertarget{interpreting-an-image-classification}{%
\section{Interpreting an Image:
Classification}\label{interpreting-an-image-classification}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-3}{%
\subsubsection*{Author}\label{author-3}}

Andréa Puzzi Nicolau, Karen Dyson, David Saah, Nicholas Clinton

\hypertarget{overview-5}{%
\subsubsection*{Overview}\label{overview-5}}

Image classification is a fundamental goal of remote sensing. It takes
the user from viewing an image to labeling its contents. This chapter
introduces readers to the concept of classification and walks users
through the many options for image classification in Earth Engine. You
will explore the processes of training data collection, classifier
selection, classifier training, and image classification.

\hypertarget{learning-outcomes-5}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-5}}

\begin{itemize}
\tightlist
\item
  Running a classification in Earth Engine.
\item
  Understanding the difference between supervised and unsupervised
  classification.
\item
  Learning how to use Earth Engine geometry drawing tools.
\item
  Learning how to collect sample data in Earth Engine.
\item
  Learning the basics of the hexadecimal numbering system.
\end{itemize}

\hypertarget{assumes-you-know-how-to-5}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-5}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Understand bands and how to select them (Chap. F1.2, Chap. F2.0).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-4}{%
\subsection*{Introduction}\label{introduction-4}}

Classification is addressed in a broad range of fields, including
mathematics, statistics, data mining, machine learning, and more. For a
deeper treatment of classification, interested readers may see some of
the following suggestions: Witten et al.~(2011), Hastie et al.~(2009),
Goodfellow et al.~(2016), Gareth et al.~(2013), Géron (2019), Müller et
al.~(2016), or Witten et al.~(2005). Unlike regression, which predicts
continuous variables, classification predicts categorical, or discrete,
variables---variables with a finite number of categories (e.g., age
range).

In remote sensing, image classification is an attempt to categorize all
pixels in an image into a finite number of labeled land cover and/or
land use classes. The resulting classified image is a simplified
thematic map derived from the original image (Fig. F2.1.1). Land cover
and land use information is essential for many environmental and
socioeconomic applications, including natural resource management, urban
planning, biodiversity conservation, agricultural monitoring, and carbon
accounting.

\begin{figure}

{\centering \includegraphics{./F2/image48.png}

}

\caption{Fig. F2.1.1 Image classification concept}

\end{figure}

Image classification techniques for generating land cover and land use
information have been in use since the 1980s (Li et al.~2014). Here, we
will cover the concepts of pixel-based supervised and unsupervised
classifications, testing out different classifiers. Chapter F3.3 covers
the concept and application of object-based classification.

It is important to define land use and land cover. Land cover relates to
the physical characteristics of the surface: simply put, it documents
whether an area of the Earth's surface is covered by forests, water,
impervious surfaces, etc. Land use refers to how this land is being used
by people. For example, herbaceous vegetation is considered a land cover
but can indicate different land uses: the grass in a pasture is an
agricultural land use, whereas the grass in an urban area can be
classified as a park.

\hypertarget{supervised-classification}{%
\subsection{Supervised Classification}\label{supervised-classification}}

If you have not already done so, be sure to add the book's code
repository to the Code Editor by entering
\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458829866098\&usg=AOvVaw16x5swm9HlorS5Mbw7E42X}{}\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458829866485\&usg=AOvVaw0-N-JCWWgnM493BKa7Ichm}{https://code.earthengine.google.com/?accept\_repo=projects/gee-edu/book}
into your browser. The book's scripts will then be available in the
script manager panel. If you have trouble finding the repo, you can
visit
\href{https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit\%23slide\%3Did.g18a7b4b055d_0_624\&sa=D\&source=editors\&ust=1671458829866823\&usg=AOvVaw0ytMyRvutssBcVr2GdcBHA}{this
link} for help.

Supervised classification uses a training dataset with known labels and
representing the spectral characteristics of each land cover class of
interest to ``supervise'' the classification. The overall approach of a
supervised classification in Earth Engine is summarized as follows:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Get a scene.
\item
  Collect training data.
\item
  Select and train a classifier using the training data.
\item
  Classify the image using the selected classifier.
\end{enumerate}

We will begin by creating training data manually, based on a clear
Landsat image (Fig. F2.1.2). Copy the code block below to define your
Landsat 8 scene variable and add it to the map. We will use a point in
Milan, Italy, as the center of the area for our image classification.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create an Earth Engine Point object over Milan.  }
\KeywordTok{var}\NormalTok{ pt }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\FloatTok{9.453}\OperatorTok{,} \FloatTok{45.424}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Filter the Landsat 8 collection and select the least cloudy image.  }
\KeywordTok{var}\NormalTok{ landsat }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(pt)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2019{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{sort}\NormalTok{(}\StringTok{\textquotesingle{}CLOUD\_COVER\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Center the map on that image.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(landsat}\OperatorTok{,} \DecValTok{8}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add Landsat image to the map.  }
\KeywordTok{var}\NormalTok{ visParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{7000}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{12000}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsat}\OperatorTok{,}\NormalTok{ visParams}\OperatorTok{,} \StringTok{\textquotesingle{}Landsat 8 image\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image44.png}

}

\caption{Fig. F2.1.2 Landsat image}

\end{figure}

Using the Geometry Tools, we will create points on the Landsat image
that represent land cover classes of interest to use as our training
data. We'll need to do two things: (1) identify where each land cover
occurs on the ground, and (2) label the points with the proper class
number. For this exercise, we will use the classes and codes shown in
Table 2.1.1.

Table 2.1.1 Land cover classes

Class

Class code

Forest

0

Developed

1

Water

2

Herbaceous

3

In the Geometry Tools, click on the marker option (Fig. F2.1.3). This
will create a point geometry which will show up as an import named
``geometry''. Click on the gear icon to configure this import.

\begin{figure}

{\centering \includegraphics{./F2/image22.png}

}

\caption{Fig. F2.1.3 Creating a new layer in the Geometry Imports}

\end{figure}

We will start by collecting forest points, so name the import forest.
Import it as a FeatureCollection, and then click + Property. Name the
new property ``class'' and give it a value of 0 (Fig. F2.1.4). We can
also choose a color to represent this class. For a forest class, it is
natural to choose a green color. You can choose the color you prefer by
clicking on it, or, for more control, you can use a hexadecimal value.

Hexadecimal values are used throughout the digital world to represent
specific colors across computers and operating systems. They are
specified by six values arranged in three pairs, with one pair each for
the red, green, and blue brightness values. If you're unfamiliar with
hexadecimal values, imagine for a moment that colors were specified in
pairs of base 10 numbers instead of pairs of base 16. In that case, a
bright pure red value would be ``990000''; a bright pure green value
would be ``009900''; and a bright pure blue value would be ``000099''. A
value like ``501263'' would be a mixture of the three colors, not
especially bright, having roughly equal amounts of blue and red, and
much less green: a color that would be a shade of purple. To create
numbers in the hexadecimal system, which might feel entirely natural if
humans had evolved to have 16 fingers, sixteen ``digits'' are needed: a
base 16 counter goes 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F,
then 10, 11, and so on. Given that counting framework, the number ``FF''
is like ``99'' in base 10: the largest two-digit number. The hexadecimal
color used for coloring the letters of the word FeatureCollection in
this book, a color with roughly equal amounts of blue and red, and much
less green, is ``7F1FA2''

Returning to the coloring of the forest points, the hexadecimal value
``589400'' is a little bit of red, about twice as much green, and no
blue: the deep green seen in Figure F2.1.4. Enter that value, with or
without the ``\#'' in front, and click OK after finishing the
configuration.

\begin{figure}

{\centering \includegraphics{./F2/image36.png}

}

\caption{Fig. F2.1.4 Edit geometry layer properties}

\end{figure}

Now, in the Geometry Imports, we will see that the import has been
renamed forest. Click on it to activate the drawing mode (Fig. F2.1.5)
in order to start collecting forest points.

\begin{figure}

{\centering \includegraphics{./F2/image29.png}

}

\caption{Fig. F2.1.5 Activate forest layer to start collection}

\end{figure}

Now, start collecting points over forested areas (Fig. F2.1.6). Zoom in
and out as needed. You can use the satellite basemap to assist you, but
the basis of your collection should be the Landsat image. Remember that
the more points you collect, the more the classifier will learn from the
information you provide. For now, let's set a goal to collect 25 points
per class. Click Exit next to Point drawing (Fig. F2.1.5) when finished.

\begin{figure}

{\centering \includegraphics{./F2/image38.png}

}

\caption{Fig. F2.1.6 Forest points}

\end{figure}

Repeat the same process for the other classes by creating new layers
(Fig. F2.1.7). Don't forget to import using the FeatureCollection option
as mentioned above. For the developed class, collect points over urban
areas. For the water class, collect points over the Ligurian Sea, and
also look for other bodies of water, like rivers. For the herbaceous
class, collect points over agricultural fields. Remember to set the
``class'' property for each class to its corresponding code (see Table
2.1.1) and click Exit once you finalize collecting points for each class
as mentioned above. We will be using the following hexadecimal colors
for the other classes: \#FF0000 for developed, \#1A11FF for water, and
\#D0741E for herbaceous.

\begin{figure}

{\centering \includegraphics{./F2/image41.png}

}

\caption{Fig. F2.1.7 New layer option in Geometry Imports}

\end{figure}

You should now have four FeatureCollection imports named forest,
developed, water, and herbaceous (Fig. F2.1.8).

\begin{figure}

{\centering \includegraphics{./F2/image42.png}

}

\caption{Fig. F2.1.8 Example of training points}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F21a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

If you wish to have the exact same results demonstrated in this chapter
from now on, continue beginning with this Code Checkpoint. If you use
the points collected yourself, the results may vary from this point
forward.

The next step is to combine all the training feature collections into
one. Copy and paste the code below to combine them into one
FeatureCollection called trainingFeatures. Here, we use the flatten
method to avoid having a collection of feature collections---we want
individual features within our FeatureCollection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Combine training feature collections.  }
\KeywordTok{var}\NormalTok{ trainingFeatures }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{([  }
\NormalTok{   forest}\OperatorTok{,}\NormalTok{ developed}\OperatorTok{,}\NormalTok{ water}\OperatorTok{,}\NormalTok{ herbaceous  }
\NormalTok{])}\OperatorTok{.}\FunctionTok{flatten}\NormalTok{()}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note: Alternatively, you could use an existing set of reference data.
For example, the European Space Agency (ESA) WorldCover dataset is a
global map of land use and land cover derived from ESA's Sentinel-2
imagery at 10 m resolution. With existing datasets, we can randomly
place points on pixels classified as the classes of interest (if you are
curious, you can explore the Earth Engine documentation to learn about
the ee.Image.stratifiedSample and the ee.FeatureCollection.randomPoints
methods). The drawback is that these global datasets will not always
contain the specific classes of interest for your region, or may not be
entirely accurate at the local scale. Another option is to use samples
that were collected in the field (e.g., GPS points). In Chap. F5.0, you
will see how to upload your own data as Earth Engine assets.

In the combined FeatureCollection, each Feature point should have a
property called ``class''. The class values are consecutive integers
from 0 to 3 (you could verify that this is true by printing
trainingFeatures and checking the properties of the features).

Now that we have our training points, copy and paste the code below to
extract the band information for each class at each point location.
First, we define the prediction bands to extract different spectral and
thermal information from different bands for each class. Then, we use
the sampleRegions method to sample the information from the Landsat
image at each point location. This method requires information about the
FeatureCollection (our reference points), the property to extract
(``class''), and the pixel scale (in meters).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define prediction bands.  }
\KeywordTok{var}\NormalTok{ predictionBands }\OperatorTok{=}\NormalTok{ [   }\StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}ST\_B10\textquotesingle{}}  
\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Sample training points.  }
\KeywordTok{var}\NormalTok{ classifierTraining }\OperatorTok{=}\NormalTok{ landsat}\OperatorTok{.}\FunctionTok{select}\NormalTok{(predictionBands)  }
   \OperatorTok{.}\FunctionTok{sampleRegions}\NormalTok{(\{  }
       \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ trainingFeatures}\OperatorTok{,}  
       \DataTypeTok{properties}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}class\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
       \DataTypeTok{scale}\OperatorTok{:} \DecValTok{30}\NormalTok{   \})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

You can check whether the classifierTraining object extracted the
properties of interest by printing it and expanding the first feature.
You should see the band and class information (Fig. F2.1.9).

\begin{figure}

{\centering \includegraphics{./F2/image20.png}

}

\caption{Fig. F2.1.9 Example of extracted band information for one point
of class 0 (forest)}

\end{figure}

Now we can choose a classifier. The choice of classifier is not always
obvious, and there are many options from which to pick---you can quickly
expand the ee.Classifier object under Docs to get an idea of how many
options we have for image classification. Therefore, we will be testing
different classifiers and comparing their results. We will start with a
Classification and Regression Tree (CART) classifier, a well-known
classification algorithm (Fig. F2.1.10) that has been around for
decades.

\begin{figure}

{\centering \includegraphics{./F2/image25.png}

}

\caption{Fig. F2.1.10 Example of a decision tree for satellite image
classification. Values and classes are hypothetical.}

\end{figure}

Copy and paste the code below to instantiate a CART classifier
(ee.Classifier.smileCart) and train it.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{//////////////// CART Classifier ///////////////////  }
  
\CommentTok{// Train a CART Classifier.  }
\KeywordTok{var}\NormalTok{ classifier }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Classifier}\OperatorTok{.}\FunctionTok{smileCart}\NormalTok{()}\OperatorTok{.}\FunctionTok{train}\NormalTok{(\{  }
   \DataTypeTok{features}\OperatorTok{:}\NormalTok{ classifierTraining}\OperatorTok{,}  
   \DataTypeTok{classProperty}\OperatorTok{:} \StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{inputProperties}\OperatorTok{:}\NormalTok{ predictionBands  }
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Essentially, the classifier contains the mathematical rules that link
labels to spectral information. If you print the variable classifier and
expand its properties, you can confirm the basic characteristics of the
object (bands, properties, and classifier being used). If you print
classifier.explain, you can find a property called ``tree'' that
contains the decision rules.

After training the classifier, copy and paste the code below to classify
the Landsat image and add it to the Map.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Classify the Landsat image.  }
\KeywordTok{var}\NormalTok{ classified }\OperatorTok{=}\NormalTok{ landsat}\OperatorTok{.}\FunctionTok{select}\NormalTok{(predictionBands)}\OperatorTok{.}\FunctionTok{classify}\NormalTok{(classifier)}\OperatorTok{;}  
  
\CommentTok{// Define classification image visualization parameters.  }
\KeywordTok{var}\NormalTok{ classificationVis }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}589400\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ff0000\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1a11ff\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}d0741e\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Add the classified image to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(classified}\OperatorTok{,}\NormalTok{ classificationVis}\OperatorTok{,} \StringTok{\textquotesingle{}CART classified\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that, in the visualization parameters, we define a palette
parameter which in this case represents colors for each pixel value
(0--3, our class codes). We use the same hexadecimal colors used when
creating our training points for each class. This way, we can associate
a color with a class when visualizing the classified image in the Map.

Inspect the result: Activate the Landsat composite layer and the
satellite basemap to overlay with the classified images (Fig. F2.1.11).
Change the layers' transparency to inspect some areas. What do you
notice? The result might not look very satisfactory in some areas (e.g.,
confusion between developed and herbaceous classes). Why do you think
this is happening? There are a few options to handle misclassification
errors:

\begin{itemize}
\tightlist
\item
  Collect more training data We can try incorporating more points to
  have a more representative sample of the classes.
\item
  Tune the model Classifiers typically have ``hyperparameters,'' which
  are set to default values. In the case of classification trees, there
  are ways to tune the number of leaves in the tree, for example. Tuning
  models is addressed in Chap. F2.2.
\item
  Try other classifiers If a classifier's results are unsatisfying, we
  can try some of the other classifiers in Earth Engine to see if the
  result is better or different.
\item
  Expand the collection location It is good practice to collect points
  across the entire image and not just focus on one location. Also, look
  for pixels of the same class that show variability (e.g., for the
  developed class, building rooftops look different than house rooftops;
  for the herbaceous class, crop fields show distinctive
  seasonality/phenology).
\item
  Add more predictors We can try adding spectral indices to the input
  variables; this way, we are feeding the classifier new, unique
  information about each class. For example, there is a good chance that
  a vegetation index specialized for detecting vegetation health (e.g.,
  NDVI) would improve the developed versus herbaceous classification.
\end{itemize}

\begin{figure}

{\centering \includegraphics{./F2/image21.png}

}

\caption{Fig. F2.1.11 CART classification}

\end{figure}

For now, we will try another supervised learning classifier that is
widely used: Random Forests (RF). The RF algorithm (Breiman 2001, Pal
2005) builds on the concept of decision trees, but adds strategies to
make them more powerful. It is called a ``forest'' because it operates
by constructing a multitude of decision trees. As mentioned previously,
a decision tree creates the rules which are used to make decisions. A
Random Forest will randomly choose features and make observations, build
a forest of decision trees, and then use the full set of trees to
estimate the class. It is a great choice when you do not have a lot of
insight about the training data.

\begin{figure}

{\centering \includegraphics{./F2/image27.png}

}

\caption{Fig. F2.1.12 General concept of Random Forests}

\end{figure}

Copy and paste the code below to train the RF classifier
(ee.Classifier.smileRandomForest) and apply the classifier to the image.
The RF algorithm requires, as its argument, the number of trees to
build. We will use 50 trees.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{/////////////// Random Forest Classifier /////////////////////  }
  
\CommentTok{// Train RF classifier.  }
\KeywordTok{var}\NormalTok{ RFclassifier }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Classifier}\OperatorTok{.}\FunctionTok{smileRandomForest}\NormalTok{(}\DecValTok{50}\NormalTok{)}\OperatorTok{.}\FunctionTok{train}\NormalTok{(\{  }
   \DataTypeTok{features}\OperatorTok{:}\NormalTok{ classifierTraining}\OperatorTok{,}  
   \DataTypeTok{classProperty}\OperatorTok{:} \StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{inputProperties}\OperatorTok{:}\NormalTok{ predictionBands  }
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Classify Landsat image.  }
\KeywordTok{var}\NormalTok{ RFclassified }\OperatorTok{=}\NormalTok{ landsat}\OperatorTok{.}\FunctionTok{select}\NormalTok{(predictionBands)}\OperatorTok{.}\FunctionTok{classify}\NormalTok{(  }
\NormalTok{   RFclassifier)}\OperatorTok{;}  
  
\CommentTok{// Add classified image to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(RFclassified}\OperatorTok{,}\NormalTok{ classificationVis}\OperatorTok{,} \StringTok{\textquotesingle{}RF classified\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that in the ee.Classifier.smileRandomForest documentation (Docs
tab), there is a seed (random number) parameter. Setting a seed allows
you to exactly replicate your model each time you run it. Any number is
acceptable as a seed.

Inspect the result (Fig. F2.1.13). How does this classified image differ
from the CART one? Is the classifications better or worse? Zoom in and
out and change the transparency of layers as needed. In Chap. F2.2, you
will see more systematic ways to assess what is better or worse, based
on accuracy metrics.

\begin{figure}

{\centering \includegraphics{./F2/image34.png}

}

\caption{Fig. F2.1.13 Random Forest classified image}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F21b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{unsupervised-classification}{%
\subsection{Unsupervised
Classification}\label{unsupervised-classification}}

In an unsupervised classification, we have the opposite process of
supervised classification. Spectral classes are grouped first and then
categorized into clusters. Therefore, in Earth Engine, these classifiers
are ee.Clusterer objects. They are ``self-taught'' algorithms that do
not use a set of labeled training data (i.e., they are
``unsupervised''). You can think of it as performing a task that you
have not experienced before, starting by gathering as much information
as possible. For example, imagine learning a new language without
knowing the basic grammar, learning only by watching a TV series in that
language, listening to examples, and finding patterns.

Similar to the supervised classification, unsupervised classification in
Earth Engine has this workflow:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Assemble features with numeric properties in which to find clusters
  (training data).
\item
  Select and instantiate a clusterer.
\item
  Train the clusterer with the training data.
\item
  Apply the clusterer to the scene (classification).
\item
  Label the clusters.
\end{enumerate}

In order to generate training data, we will use the sample method, which
randomly takes samples from a region (unlike sampleRegions, which takes
samples from predefined locations). We will use the image's footprint as
the region by calling the geometry method. Additionally, we will define
the number of pixels (numPixels) to sample---in this case, 1000
pixels---and define a tileScale of 8 to avoid computation errors due to
the size of the region. Copy and paste the code below to sample 1000
pixels from the Landsat image. You should add to the same script as
before to compare supervised versus unsupervised classification results
at the end.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{//////////////// Unsupervised classification ////////////////  }
  
\CommentTok{// Make the training dataset.  }
\KeywordTok{var}\NormalTok{ training }\OperatorTok{=}\NormalTok{ landsat}\OperatorTok{.}\FunctionTok{sample}\NormalTok{(\{  }
   \DataTypeTok{region}\OperatorTok{:}\NormalTok{ landsat}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:} \DecValTok{30}\OperatorTok{,}  
   \DataTypeTok{numPixels}\OperatorTok{:} \DecValTok{1000}\OperatorTok{,}  
   \DataTypeTok{tileScale}\OperatorTok{:} \DecValTok{8}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now we can instantiate a clusterer and train it. As with the supervised
algorithms, there are many unsupervised algorithms to choose from. We
will use the k-means clustering algorithm, which is a commonly used
approach in remote sensing. This algorithm identifies groups of pixels
near each other in the spectral space (image x bands) by using an
iterative regrouping strategy. We define a number of clusters, k, and
then the method randomly distributes that number of seed points into the
spectral space. A large sample of pixels is then grouped into its
closest seed, and the mean spectral value of this group is calculated.
That mean value is akin to a center of mass of the points, and is known
as the centroid. Each iteration recalculates the class means and
reclassifies pixels with respect to the new means. This process is
repeated until the centroids remain relatively stable and only a few
pixels change from class to class on subsequent iterations.

\begin{figure}

{\centering \includegraphics{./F2/image35.png}

}

\caption{Fig. F2.1.14 K-means visual concept}

\end{figure}

Copy and paste the code below to request four clusters, the same number
as for the supervised classification, in order to directly compare them.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Instantiate the clusterer and train it.  }
\KeywordTok{var}\NormalTok{ clusterer }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Clusterer}\OperatorTok{.}\FunctionTok{wekaKMeans}\NormalTok{(}\DecValTok{4}\NormalTok{)}\OperatorTok{.}\FunctionTok{train}\NormalTok{(training)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now copy and paste the code below to apply the clusterer to the image
and add the resulting classification to the Map (Fig. F2.1.15). Note
that we are using a method called randomVisualizer to assign colors for
the visualization. We are not associating the unsupervised classes with
the color palette we defined earlier in the supervised classification.
Instead, we are assigning random colors to the classes, since we do not
yet know which of the unsupervised classes best corresponds to each of
the named classes (e.g., forest , herbaceous). Note that the colors in
Fig. F1.2.15 might not be the same as you see on your Map, since they
are assigned randomly.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Cluster the input using the trained clusterer.  }
\KeywordTok{var}\NormalTok{ Kclassified }\OperatorTok{=}\NormalTok{ landsat}\OperatorTok{.}\FunctionTok{cluster}\NormalTok{(clusterer)}\OperatorTok{;}  
  
\CommentTok{// Display the clusters with random colors.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(Kclassified}\OperatorTok{.}\FunctionTok{randomVisualizer}\NormalTok{()}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}   \StringTok{\textquotesingle{}K{-}means classified {-} random colors\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image31.png}

}

\caption{Fig. F2.1.15 K-means classification}

\end{figure}

Inspect the results. How does this classification compare to the
previous ones? If preferred, use the Inspector to check which classes
were assigned to each pixel value (``cluster'' band) and change the last
line of your code to apply the same palette used for the supervised
classification results (see Code Checkpoint below for an example).

Another key point of classification is the accuracy assessment of the
results. This will be covered in Chap. F2.2.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F21c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-5}{%
\subsection*{Conclusion}\label{conclusion-5}}
\addcontentsline{toc}{subsection}{Conclusion}

Classification algorithms are key for many different applications
because they allow you to predict categorical variables. You should now
understand the difference between supervised and unsupervised
classification and have the basic knowledge on how to handle
misclassifications. By being able to map the landscape for land use and
land cover, we will also be able to monitor how it changes (Part F4).

\hypertarget{references-3}{%
\subsection*{References}\label{references-3}}
\addcontentsline{toc}{subsection}{References}

Breiman L (2001) Random forests. Mach Learn 45:5--32.
https://doi.org/10.1023/A:1010933404324

Gareth J, Witten D, Hastie T, Tibshirani R (2013) An Introduction to
Statistical Learning. Springer

Géron A (2019) Hands-on Machine Learning with Scikit-Learn, Keras and
TensorFlow: Concepts, Tools, and Techniques to Build Intelligent
Systems. O'Reilly Media, Inc.

Goodfellow I, Bengio Y, Courville A (2016) Deep Learning. MIT Press

Hastie T, Tibshirani R, Friedman JH (2009) The Elements of Statistical
Learning: Data Mining, Inference, and Prediction. Springer

Li M, Zang S, Zhang B, et al (2014) A review of remote sensing image
classification techniques: The role of spatio-contextual information.
Eur J Remote Sens 47:389--411. https://doi.org/10.5721/EuJRS20144723

Müller AC, Guido S (2016) Introduction to Machine Learning with Python:
A Guide for Data Scientists. O'Reilly Media, Inc.

Pal M (2005) Random forest classifier for remote sensing classification.
Int J Remote Sens 26:217--222.
https://doi.org/10.1080/01431160412331269698

Witten IH, Frank E, Hall MA, et al (2005) Practical machine learning
tools and techniques. In: Data Mining. pp 4

\hypertarget{accuracy-assessment-quantifying-classification-quality}{%
\section{Accuracy Assessment: Quantifying Classification
Quality}\label{accuracy-assessment-quantifying-classification-quality}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-4}{%
\subsubsection*{Author}\label{author-4}}

Andréa Puzzi Nicolau, Karen Dyson, David Saah, Nicholas Clinton

\hypertarget{overview-6}{%
\subsubsection*{Overview}\label{overview-6}}

This chapter will enable you to assess the accuracy of an image
classification. You will learn about different metrics and ways to
quantify classification quality in Earth Engine. Upon completion, you
should be able to evaluate whether your classification needs improvement
and know how to proceed when it does.

\hypertarget{learning-outcomes-6}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-6}}

\begin{itemize}
\tightlist
\item
  Learning how to perform accuracy assessment in Earth Engine.
\item
  Understanding how to generate and read a confusion matrix.
\item
  Understanding overall accuracy and the kappa coefficient.
\item
  Understanding the difference between user's and producer's accuracy,
  and the difference between omission and commission errors.
\end{itemize}

\hypertarget{assumes-you-know-how-to-6}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-6}}

\begin{itemize}
\tightlist
\item
  \hspace{0pt}\hspace{0pt}Create a graph using ui.Chart (Chap. F1.3).
\item
  Perform a supervised Random Forest image classification (Chap. F2.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-5}{%
\subsection*{Introduction}\label{introduction-5}}

Any map or remotely sensed product is a generalization or model that
will have inherent errors. Products derived from remotely sensed data
used for scientific purposes and policymaking require a quantitative
measure of accuracy to strengthen the confidence in the information
generated (Foody 2002, Strahler et al.~2006, Olofsson et al.~2014).
Accuracy assessment is a crucial part of any classification project, as
it measures the degree to which the classification agrees with another
data source that is considered to be accurate, ground-truth data (i.e.,
``reality'').

The history of accuracy assessment reveals increasing detail and rigor
in the analysis, moving from a basic visual appraisal of the derived map
(Congalton 1994, Foody 2002) to the definition of best practices for
sampling and response designs and the calculation of accuracy metrics
(Foody 2002, Stehman 2013, Olofsson et al.~2014, Stehman and Foody
2019). The confusion matrix (also called the ``error matrix'') (Stehman
1997) summarizes key accuracy metrics used to assess products derived
from remotely sensed data.

In Chap. F2.1, we asked whether the classification results were
satisfactory. In remote sensing, the quantification of the answer to
that question is called accuracy assessment. In the classification
context, accuracy measurements are often derived from a confusion
matrix.

In a thorough accuracy assessment, we think carefully about the sampling
design, the response design, and the analysis (Olofsson et al.~2014).
Fundamental protocols are taken into account to produce scientifically
rigorous and transparent estimates of accuracy and area, which requires
robust planning and time. In a standard setting, we would calculate the
number of samples needed for measuring accuracy (sampling design). Here,
we will focus mainly on the last step, analysis, by examining the
confusion matrix and learning how to calculate the accuracy metrics.
This will be done by partitioning the existing data into training and
testing sets.

\hypertarget{quantifying-classification-accuracy-through-a-confusion-matrix}{%
\subsection{Quantifying Classification Accuracy Through a Confusion
Matrix}\label{quantifying-classification-accuracy-through-a-confusion-matrix}}

If you have not already done so, be sure to add the book's code
repository to the Code Editor by entering
\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458829937499\&usg=AOvVaw3qqOwSX_A-Pllh6X3X31q4}{}\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458829937976\&usg=AOvVaw0WioXIhzue8-WoaX4UtabH}{https://code.earthengine.google.com/?accept\_repo=projects/gee-edu/book}
into your browser. The book's scripts will then be available in the
script manager panel. If you have trouble finding the repo, you can
visit
\href{https://www.google.com/url?q=https://docs.google.com/presentation/d/1Kt6wGNoesYm__Cu3k3bnlbbyPN6m9SF4hQHK-pIDHfc/edit\%23slide\%3Did.g18a7b4b055d_0_624\&sa=D\&source=editors\&ust=1671458829938470\&usg=AOvVaw2CH8V3-_qV99EcgMxUAaSO}{this
link} for help.

To illustrate some of the basic ideas about classification accuracy, we
will revisit the data and location of part of Chap. F2.1, where we
tested different classifiers and classified a Landsat image of the area
around Milan, Italy. We will name this dataset `data'. This variable is
a FeatureCollection with features containing the ``class'' values (Table
F2.2.1) and spectral information of four land cover / land use classes:
forest, developed, water, and herbaceous (see Fig. F2.1.8 and Fig.
F2.1.9 for a refresher). We will also define a variable,
predictionBands, which is a list of bands that will be used for
prediction (classification)---the spectral information in the data
variable.

Table F2.2.1 Land cover classes

Class

Class value

Forest

0

Developed

1

Water

2

Herbaceous

3

The first step is to partition the set of known values into training and
testing sets in order to have something for the classifier to predict
over that it has not been shown before (the testing set), mimicking
unseen data that the model might see in the future. We add a column of
random numbers to our FeatureCollection using the randomColumn method.
Then, we filter the features into about 80\% for training and 20\% for
testing using ee.Filter. Copy and paste the code below to partition the
data and filter features based on the random number.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the reference dataset.  }
\KeywordTok{var}\NormalTok{ data }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(   }\StringTok{\textquotesingle{}projects/gee{-}book/assets/F2{-}2/milan\_data\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Define the prediction bands.  }
\KeywordTok{var}\NormalTok{ predictionBands }\OperatorTok{=}\NormalTok{ [   }\StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}ST\_B10\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}ndvi\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ndwi\textquotesingle{}}  
\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Split the dataset into training and testing sets.  }
\KeywordTok{var}\NormalTok{ trainingTesting }\OperatorTok{=}\NormalTok{ data}\OperatorTok{.}\FunctionTok{randomColumn}\NormalTok{()}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ trainingSet }\OperatorTok{=}\NormalTok{ trainingTesting  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{lessThan}\NormalTok{(}\StringTok{\textquotesingle{}random\textquotesingle{}}\OperatorTok{,} \FloatTok{0.8}\NormalTok{))}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ testingSet }\OperatorTok{=}\NormalTok{ trainingTesting  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{greaterThanOrEquals}\NormalTok{(}\StringTok{\textquotesingle{}random\textquotesingle{}}\OperatorTok{,} \FloatTok{0.8}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that randomColumn creates pseudorandom numbers in a deterministic
way. This makes it possible to generate a reproducible pseudorandom
sequence by defining the seed parameter (Earth Engine uses a seed of 0
by default). In other words, given a starting value (i.e., the seed),
randomColumn will always provide the same sequence of pseudorandom
numbers.

Copy and paste the code below to train a Random Forest classifier with
50 decision trees using the trainingSet.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Train the Random Forest Classifier with the trainingSet.  }
\KeywordTok{var}\NormalTok{ RFclassifier }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Classifier}\OperatorTok{.}\FunctionTok{smileRandomForest}\NormalTok{(}\DecValTok{50}\NormalTok{)}\OperatorTok{.}\FunctionTok{train}\NormalTok{(\{  }
   \DataTypeTok{features}\OperatorTok{:}\NormalTok{ trainingSet}\OperatorTok{,}  
   \DataTypeTok{classProperty}\OperatorTok{:} \StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{inputProperties}\OperatorTok{:}\NormalTok{ predictionBands  }
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now, let's discuss what a confusion matrix is. A confusion matrix
describes the quality of a classification by comparing the predicted
values to the actual values. A simple example is a confusion matrix for
a binary classification into the classes ``positive'' and ``negative,''
as shown in Table F2.2.1.

Table F2.2.1 Confusion matrix for a binary classification where the
classes are ``positive'' and ``negative''

Actual values

Positive

Negative

Predicted values

Positive

TP (true positive)

FP (false positive)

Negative

FN (false negative)

TN (true negative)

In Table F2.2.1, the columns represent the actual values (the truth),
while the rows represent the predictions (the classification). ``True
positive'' (TP) and ``true negative'' (TN) mean that the classification
of a pixel matches the truth (e.g., a water pixel correctly classified
as water). ``False positive'' (FP) and ``false negative'' (FN) mean that
the classification of a pixel does not match the truth (e.g., a
non-water pixel incorrectly classified as water).

\begin{itemize}
\tightlist
\item
  TP: classified as positive and the actual class is positive
\item
  FP: classified as positive and the actual class is negative
\item
  FN: classified as negative and the actual class is positive
\item
  TN: classified as negative and the actual class is negative
\end{itemize}

We can extract some statistical information from a confusion matrix..
Let's look at an example to make this clearer. Table F2.2.2 is a
confusion matrix for a sample of 1,000 pixels for a classifier that
identifies whether a pixel is forest (positive) or non-forest
(negative), a binary classification.

Table F2.2.2 Confusion matrix for a binary classification where the
classes are ``positive'' (forest) and ``negative'' (non-forest)

Actual values

Positive

Negative

Predicted values

Positive

307

18

Negative

14

661

In this case, the classifier correctly identified 307 forest pixels,
wrongly classified 18 non-forest pixels as forest, correctly identified
661 non-forest pixels, and wrongly classified 14 forest pixels as
non-forest. Therefore, the classifier was correct 968 times and wrong 32
times. Let's calculate the main accuracy metrics for this example.

The overall accuracy tells us what proportion of the reference data was
classified correctly, and is calculated as the total number of correctly
identified pixels divided by the total number of pixels in the sample.

\includegraphics{./F2/image6.png}

In this case, the overall accuracy is 96.8\%, calculated using
(\includegraphics{./F2/image7.png}.

Two other important accuracy metrics are the producer's accuracy and the
user's accuracy, also referred to as the ``recall'' and the
``precision,'' respectively. Importantly, these metrics quantify aspects
of per-class accuracy.

The producer's accuracy is the accuracy of the map from the point of
view of the map maker (the ``producer''), and is calculated as the
number of correctly identified pixels of a given class divided by the
total number of pixels actually in that class. The producer's accuracy
for a given class tells us the proportion of the pixels in that class
that were classified correctly.

\includegraphics{./F2/image8.png}

\includegraphics{./F2/image9.png}

In this case, the producer's accuracy for the forest class is 95.6\%,
calculated using \includegraphics{./F2/image10.png}). The producer's
accuracy for the non-forest class is 97.3\%, calculated from
\includegraphics{./F2/image11.png}).

The user's accuracy (also called the ``consumer's accuracy'') is the
accuracy of the map from the point of view of a map user, and is
calculated as the number of correctly identified pixels of a given class
divided by the total number of pixels claimed to be in that class. The
user's accuracy for a given class tells us the proportion of the pixels
identified on the map as being in that class that are actually in that
class on the ground.

\includegraphics{./F2/image12.png}

\includegraphics{./F2/image13.png}

In this case, the user's accuracy for the forest class is 94.5\%,
calculated using \includegraphics{./F2/image14.png}). The user's
accuracy for the non-forest class is 97.9\%, calculated from
\includegraphics{./F2/image15.png}).

Fig. F2.2.1 helps visualize the rows and columns used to calculate each
accuracy.

\begin{figure}

{\centering \includegraphics{./F2/image43.png}

}

\caption{Fig. F2.2.1 Confusion matrix for a binary classification where
the classes are ``positive'' (forest) and ``negative'' (non-forest),
with accuracy metrics}

\end{figure}

It is very common to talk about two types of error when addressing
remote-sensing classification accuracy: omission errors and commission
errors. Omission errors refer to the reference pixels that were left out
of (omitted from) the correct class in the classified map. In a
two-class system, an error of omission in one class will be counted as
an error of commission in another class. Omission errors are
complementary to the producer's accuracy.

\includegraphics{./F2/image16.png}

Commission errors refer to the class pixels that were erroneously
classified in the map and are complementary to the user's accuracy.

\includegraphics{./F2/image17.png}

Finally, another commonly used accuracy metric is the kappa coefficient,
which evaluates how well the classification performed as compared to
random. The value of the kappa coefficient can range from −1 to 1: a
negative value indicates that the classification is worse than a random
assignment of categories would have been; a value of 0 indicates that
the classification is no better or worse than random; and a positive
value indicates that the classification is better than random.

\includegraphics{./F2/image18.png}

The chance agreement is calculated as the sum of the product of row and
column totals for each class, and the observed accuracy is the overall
accuracy. Therefore, for our example, the kappa coefficient is 0.927.

\includegraphics{./F2/image19.png}

Now, let's go back to the script. In Earth Engine, there are API calls
for these operations. Note that our confusion matrix will be a 4 x 4
table, since we have four different classes.

Copy and paste the code below to classify the testingSet and get a
confusion matrix using the method errorMatrix. Note that the classifier
automatically adds a property called ``classification,'' which is
compared to the ``class'' property of the reference dataset.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Now, to test the classification (verify model\textquotesingle{}s accuracy),  }
\CommentTok{// we classify the testingSet and get a confusion matrix.  }
\KeywordTok{var}\NormalTok{ confusionMatrix }\OperatorTok{=}\NormalTok{ testingSet}\OperatorTok{.}\FunctionTok{classify}\NormalTok{(RFclassifier)  }
   \OperatorTok{.}\FunctionTok{errorMatrix}\NormalTok{(\{  }
       \DataTypeTok{actual}\OperatorTok{:} \StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{predicted}\OperatorTok{:} \StringTok{\textquotesingle{}classification\textquotesingle{}}\NormalTok{   \})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Copy and paste the code below to print the confusion matrix and accuracy
metrics. Expand the confusion matrix object to inspect it. The entries
represent the number of pixels. Items on the diagonal represent correct
classification. Items off the diagonal are misclassifications, where the
class in row i is classified as column j (values from 0 to 3 correspond
to our class codes: forest, developed, water, and herbaceous,
respectively). Also expand the producer's accuracy, user's accuracy
(consumer's accuracy), and kappa coefficient objects to inspect them.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Print the results.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Confusion matrix:\textquotesingle{}}\OperatorTok{,}\NormalTok{ confusionMatrix)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Overall Accuracy:\textquotesingle{}}\OperatorTok{,}\NormalTok{ confusionMatrix}\OperatorTok{.}\FunctionTok{accuracy}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Producers Accuracy:\textquotesingle{}}\OperatorTok{,}\NormalTok{ confusionMatrix}\OperatorTok{.}\FunctionTok{producersAccuracy}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Consumers Accuracy:\textquotesingle{}}\OperatorTok{,}\NormalTok{ confusionMatrix}\OperatorTok{.}\FunctionTok{consumersAccuracy}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Kappa:\textquotesingle{}}\OperatorTok{,}\NormalTok{ confusionMatrix}\OperatorTok{.}\FunctionTok{kappa}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

How is the classification accuracy? Which classes have higher accuracy
compared to the others? Can you think of any reasons why? (Hint: Check
where the errors in these classes are in the confusion matrix---i.e.,
being committed and omitted.)

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F22a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{hyperparameter-tuning}{%
\subsection{Hyperparameter tuning}\label{hyperparameter-tuning}}

We can also assess how the number of trees in the Random Forest
classifier affects the classification accuracy. Copy and paste the code
below to create a function that charts the overall accuracy versus the
number of trees used. The code tests from 5 to 100 trees at increments
of 5, producing Fig. F2.2.2. (Do not worry too much about fully
understanding each item at this stage of your learning. If you want to
find out how these operations work, you can see more in Chaps. F4.0 and
F4.1.)

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Hyperparameter tuning.  }
\KeywordTok{var}\NormalTok{ numTrees }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{List}\OperatorTok{.}\FunctionTok{sequence}\NormalTok{(}\DecValTok{5}\OperatorTok{,} \DecValTok{100}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ accuracies }\OperatorTok{=}\NormalTok{ numTrees}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(t) \{   }\KeywordTok{var}\NormalTok{ classifier }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Classifier}\OperatorTok{.}\FunctionTok{smileRandomForest}\NormalTok{(t)  }
       \OperatorTok{.}\FunctionTok{train}\NormalTok{(\{  }
           \DataTypeTok{features}\OperatorTok{:}\NormalTok{ trainingSet}\OperatorTok{,}  
           \DataTypeTok{classProperty}\OperatorTok{:} \StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,}  
           \DataTypeTok{inputProperties}\OperatorTok{:}\NormalTok{ predictionBands  }
\NormalTok{       \})}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ testingSet  }
       \OperatorTok{.}\FunctionTok{classify}\NormalTok{(classifier)  }
       \OperatorTok{.}\FunctionTok{errorMatrix}\NormalTok{(}\StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}classification\textquotesingle{}}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{accuracy}\NormalTok{()}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{array}\OperatorTok{.}\FunctionTok{values}\NormalTok{(\{  }
   \DataTypeTok{array}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Array}\NormalTok{(accuracies)}\OperatorTok{,}  
   \DataTypeTok{axis}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{xLabels}\OperatorTok{:}\NormalTok{ numTrees  }
\NormalTok{\})}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{  }
   \DataTypeTok{hAxis}\OperatorTok{:}\NormalTok{ \{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Number of trees\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
   \DataTypeTok{vAxis}\OperatorTok{:}\NormalTok{ \{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Accuracy\textquotesingle{}}\NormalTok{   \}}\OperatorTok{,}  
   \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Accuracy per number of trees\textquotesingle{}}  
\NormalTok{\}))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F2/image45.png}

}

\caption{Fig. F2.2.2 Chart showing accuracy per number of Random Forest
trees}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F22b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Section 3. Spatial autocorrelation

We might also want to ensure that the samples from the training set are
uncorrelated with the samples from the testing set. This might result
from the spatial autocorrelation of the phenomenon being predicted. One
way to exclude samples that might be correlated in this manner is to
remove samples that are within some distance to any other sample. In
Earth Engine, this can be accomplished with a spatial join. The
following Code Checkpoint replicates Sect. 1 but with a spatial join
that excludes training points that are less than 1000 meters distant
from testing points.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F22c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-6}{%
\subsection*{Conclusion}\label{conclusion-6}}
\addcontentsline{toc}{subsection}{Conclusion}

You should now understand how to calculate how well your classifier is
performing on the data used to build the model. This is a useful way to
understand how a classifier is performing, because it can help indicate
which classes are performing better than others. A poorly modeled class
can sometimes be improved by, for example, collecting more training
points for that class.

Nevertheless, a model may work well on training data but work poorly in
locations randomly chosen in the study area. To understand a model's
behavior on testing data, analysts employ protocols required to produce
scientifically rigorous and transparent estimates of the accuracy and
area of each class in the study region. We will not explore those
practices in this chapter, but if you are interested, there are
tutorials and papers available online that can guide you through the
process. Links to some of those tutorials can be found in the ``For
Further Reading'' section of this book.

References

Congalton R (1994) Accuracy assessment of remotely sensed data: Future
needs and directions. In: Proceedings of Pecora 12 land information from
space-based systems. pp 385--388

Foody GM (2002) Status of land cover classification accuracy assessment.
Remote Sens Environ 80:185--201.
https://doi.org/10.1016/S0034-4257(01)00295-4

Olofsson P, Foody GM, Herold M, et al (2014) Good practices for
estimating area and assessing accuracy of land change. Remote Sens
Environ 148:42--57. https://doi.org/10.1016/j.rse.2014.02.015

Stehman SV (2013) Estimating area from an accuracy assessment error
matrix. Remote Sens Environ 132:202--211.
https://doi.org/10.1016/j.rse.2013.01.016

Stehman SV (1997) Selecting and interpreting measures of thematic
classification accuracy. Remote Sens Environ 62:77--89.
https://doi.org/10.1016/S0034-4257(97)00083-7

Stehman SV, Foody GM (2019) Key issues in rigorous accuracy assessment
of land cover products. Remote Sens Environ 231:111199.
https://doi.org/10.1016/j.rse.2019.05.018

Strahler AH, Boschetti L, Foody GM, et al (2006) Global land cover
validation: Recommendations for evaluation and accuracy assessment of
global land cover maps. Eur Communities, Luxemb 51:1--60

\hypertarget{interpreting-image-series}{%
\chapter{Interpreting Image Series}\label{interpreting-image-series}}

One of the paradigm-changing features of Earth Engine is the ability to
access decades of imagery without the previous limitation of needing to
download all the data to a local disk for processing. Because
remote-sensing data files can be enormous, this used to limit many
projects to viewing two or three images from different periods. With
Earth Engine, users can access tens or hundreds of thousands of images
to understand the status of places across decades.

\hypertarget{filter-map-reduce}{%
\section{Filter, Map, Reduce}\label{filter-map-reduce}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-5}{%
\subsubsection*{Author}\label{author-5}}

Jeffrey A. Cardille

\hypertarget{overview-7}{%
\subsubsection*{Overview}\label{overview-7}}

The purpose of this chapter is to teach you important programming
concepts as they are applied in Earth Engine. We first illustrate how
the order and type of these operations can matter with a real-world,
non-programming example. We then demonstrate these concepts with an
ImageCollection, a key data type that distinguishes Earth Engine from
desktop image-processing implementations.

\hypertarget{learning-outcomes-7}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-7}}

\begin{itemize}
\tightlist
\item
  Visualizing the concepts of filtering, mapping, and reducing with a
  hypothetical, non-programming example.
\item
  Gaining context and experience with filtering an ImageCollection.
\item
  Learning how to efficiently map a user-written function over the
  images of a filtered ImageCollection.
\item
  Learning how to summarize a set of assembled values using Earth Engine
  reducers.
\end{itemize}

\hypertarget{assumes-you-know-how-to-7}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-7}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-6}{%
\subsection*{Introduction}\label{introduction-6}}

Prior chapters focused on exploring individual images---for example,
viewing the characteristics of single satellite images by displaying
different combinations of bands (Chap. F1.1), viewing single images from
different datasets (Chap. F1.2, Chap. F1.3), and exploring image
processing principles (Parts F2, F3) as they are implemented for
cloud-based remote sensing in Earth Engine. Each image encountered in
those chapters was pulled from a larger assemblage of images taken from
the same sensor. The chapters used a few ways to narrow down the number
of images in order to view just one for inspection (Part F1) or
manipulation (Part F2, Part F3).

In this chapter and most of the chapters that follow, we will move from
the domain of single images to the more complex and distinctive world of
working with image collections, one of the fundamental data types within
Earth Engine. The ability to conceptualize and manipulate entire image
collections distinguishes Earth Engine and gives it considerable power
for interpreting change and stability across space and time.

When looking for change or seeking to understand differences in an area
through time, we often proceed through three ordered stages, which we
will color code in this first explanatory part of the lab:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Filter: selecting subsets of images based on criteria of interest.
\item
  Map: manipulating each image in a set in some way to suit our goals.
  and
\item
  Reduce: estimating characteristics of the time series.
\end{enumerate}

For users of other programming languages---R, MATLAB, C, Karel, and many
others---this approach might seem awkward at first. We explain it below
with a non-programming example: going to the store to buy milk.

Suppose you need to go shopping for milk, and you have two criteria for
determining where you will buy your milk: location and price. The store
needs to be close to your home, and as a first step in deciding whether
to buy milk today, you want to identify the lowest price among those
stores. You don't know the cost of milk at any store ahead of time, so
you need to efficiently contact each one and determine the minimum price
to know whether it fits in your budget. If we were discussing this with
a friend, we might say, ``I need to find out how much milk costs at all
the stores around here.'' To solve that problem in a programming
language, these words imply precise operations on sets of information.
We can write the following ``pseudocode,'' which uses words that
indicate logical thinking but that cannot be pasted directly into a
program:

AllStoresOnEarth.filterNearbyStores.filterStoresWithMilk.getMilkPricesFromEachStore.determineTheMinimumValue

Imagine doing these actions not on a computer but in a more
old-fashioned way: calling on the telephone for milk prices, writing the
milk prices on paper, and inspecting the list to find the lowest value.
In this approach, we begin with AllStoresOnEarth, since there is at
least some possibility that we could decide to visit any store on Earth,
a set that could include millions of stores, with prices for millions or
billions of items. A wise first action would be to limit ourselves to
nearby stores. Asking to filterNearbyStores would reduce the number of
potential stores to hundreds, depending on how far we are willing to
travel for milk. Then, working with that smaller set, we further
filterStoresWithMilk, limiting ourselves to stores that sell our target
item. At that point in the filtering, imagine that just 10 possibilities
remain. Then, by telephone, we getMilkPricesFromEachStore, making a
short paper list of prices. We then scan the list to
determineTheMinimumValue to decide which store to visit.

In that example, each color plays a different role in the workflow. The
AllStoresOnEarth set, any one of which might contain inexpensive milk,
is an enormous collection. The filtering actions filterNearbyStores and
filterStoresWithMilk are operations that can happen on any set of
stores. These actions take a set of stores, do some operation to limit
that set, and return that smaller set of stores as an answer. The action
to getMilkPricesFromEachStore takes a simple idea---calling a store for
a milk price---and ``maps'' it over a given set of stores. Finally, with
the list of nearby milk prices assembled, the action to
determineTheMinimumValue, a general idea that could be applied to any
list of numbers, identifies the cheapest one.

The list of steps above might seem almost too obvious, but the choice
and order of operations can have a big impact on the feasibility of the
problem. Imagine if we had decided to do the same operations in a
slightly different order:

AllStoresOnEarth.filterStoresWithMilk.getMilkPricesFromEachStore.filterNearbyStores.determineMinimumValue

In this approach, we first identify all the stores on Earth that have
milk, then contact them one by one to get their current milk price. If
the contact is done by phone, this could be a painfully slow process
involving millions of phone calls. It would take considerable
``processing'' time to make each call, and careful work to record each
price onto a giant list. Processing the operations in this order would
demand that only after entirely finishing the process of contacting
every milk proprietor on Earth, we then identify the ones on our list
that are not nearby enough to visit, then scan the prices on the list of
nearby stores to find the cheapest one. This should ultimately give the
same answer as the more efficient first example, but only after
requiring so much effort that we might want to give up.

In addition to the greater order of magnitude of the list size, you can
see that there are also possible slow points in the process. Could you
make a million phone calls yourself? Maybe, but it might be pretty
appealing to hire, say, 1000 people to help. While being able to make a
large number of calls in parallel would speed up the calling stage, it's
important to note that you would need to wait for all 1000 callers to
return their sublists of prices. Why wait? Nearby stores could be on any
caller's sublist, so any caller might be the one to find the lowest
nearby price. The identification of the lowest nearby price would need
to wait for the slowest caller, even if it turned out that all of that
last caller's prices came from stores on the other side of the world.

This counterexample would also have other complications---such as the
need to track store locations on the list of milk prices---that could
present serious problems if you did those operations in that unwise
order. For now, the point is to filter, then map, then reduce. Below,
we'll apply these concepts to image collections.

\hypertarget{filtering-image-collections-in-earth-engine}{%
\subsection{Filtering Image Collections in Earth
Engine}\label{filtering-image-collections-in-earth-engine}}

The first part of the filter, map, reduce paradigm is ``filtering'' to
get a smaller ImageCollection from a larger one. As in the milk example,
filters take a large set of items, limit it by some criterion, and
return a smaller set for consideration. Here, filters take an
ImageCollection, limit it by some criterion of date, location, or image
characteristics, and return a smaller ImageCollection (Fig. F4.0.1).

\begin{figure}

{\centering \includegraphics{./F4/image66.png}

}

\caption{Fig. 4.0.1 Filter, map, reduce as applied to image collections
in Earth Engine}

\end{figure}

As described first in Chap. F1.2, the Earth Engine API provides a set of
filters for the ImageCollection type. The filters can limit an
ImageCollection based on spatial, temporal, or attribute
characteristics. Filters were used in Parts F1, F2, and F3 without much
context or explanation, to isolate an image from an ImageCollection for
inspection or manipulation. The information below should give
perspective on that work while introducing some new tools for filtering
image collections.

Below are three examples of limiting a Landsat 5 ImageCollection by
characteristics and assessing the size of the resulting set.

FilterDate This takes an ImageCollection as input and returns an
ImageCollection whose members satisfy the specified date criteria. We'll
adapt the earlier filtering logic seen in Chap. F1.2:

var imgCol = ee.ImageCollection(`LANDSAT/LT05/C02/T1\_L2');

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// How many Tier 1 Landsat 5 images have ever been collected?  }
\FunctionTok{print}\NormalTok{(}\StringTok{"All images ever: "}\OperatorTok{,}\NormalTok{ imgCol}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;} \CommentTok{// A very large number  }
  
\CommentTok{// How many images were collected in the 2000s?  }
\KeywordTok{var}\NormalTok{ startDate }\OperatorTok{=} \StringTok{\textquotesingle{}2000{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=} \StringTok{\textquotesingle{}2010{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ imgColfilteredByDate }\OperatorTok{=}\NormalTok{ imgCol}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{"All images 2000{-}2010: "}\OperatorTok{,}\NormalTok{ imgColfilteredByDate}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}  
\CommentTok{// A smaller (but still large) number}

\NormalTok{After running the code}\OperatorTok{,}\NormalTok{ you should get a very large number }\ControlFlowTok{for}\NormalTok{ the full set }\KeywordTok{of}\NormalTok{ images}\OperatorTok{.} \AttributeTok{You}\NormalTok{ also will likely get a very large number }\ControlFlowTok{for}\NormalTok{ the subset }\KeywordTok{of}\NormalTok{ images over the decade}\OperatorTok{{-}}\NormalTok{scale interval}\OperatorTok{.}  

\NormalTok{FilterBounds It may be that—similar to the milk example—only images near to a place }\KeywordTok{of}\NormalTok{ interest are useful }\ControlFlowTok{for}\NormalTok{ you}\OperatorTok{.} \AttributeTok{As}\NormalTok{ first presented }\KeywordTok{in}\NormalTok{ Part F1}\OperatorTok{,}\NormalTok{ filterBounds takes an ImageCollection }\ImportTok{as}\NormalTok{ input and returns an ImageCollection whose images surround a specified location}\OperatorTok{.} \AttributeTok{If}\NormalTok{ we take the ImageCollection that was filtered by date and then filter it by bounds}\OperatorTok{,}\NormalTok{ we will have filtered the collection to those images near a specified point within the specified date interval}\OperatorTok{.} \AttributeTok{With}\NormalTok{ the code below}\OperatorTok{,}\NormalTok{ we’ll count the number }\KeywordTok{of}\NormalTok{ images }\KeywordTok{in}\NormalTok{ the Shanghai vicinity}\OperatorTok{,}\NormalTok{ first visited }\KeywordTok{in}\NormalTok{ Chap}\OperatorTok{.} \AttributeTok{F1}\OperatorTok{.}\DecValTok{1}\OperatorTok{,} \ImportTok{from}\NormalTok{ the early }\DecValTok{2000}\ErrorTok{s}\OperatorTok{:}

\KeywordTok{var}\NormalTok{ ShanghaiImage }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(   }\StringTok{\textquotesingle{}LANDSAT/LT05/C02/T1\_L2/LT05\_118038\_20000606\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(ShanghaiImage}\OperatorTok{,} \DecValTok{9}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ imgColfilteredByDateHere }\OperatorTok{=}\NormalTok{ imgColfilteredByDate}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}   \OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{"All images here, 2000{-}2010: "}\OperatorTok{,}\NormalTok{ imgColfilteredByDateHere  }
\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;} \CommentTok{// A smaller number}

\NormalTok{If you’d like}\OperatorTok{,}\NormalTok{ you could take a few minutes to explore the behavior }\KeywordTok{of}\NormalTok{ the script }\KeywordTok{in}\NormalTok{ different parts }\KeywordTok{of}\NormalTok{ the world}\OperatorTok{.} \AttributeTok{To} \ControlFlowTok{do}\NormalTok{ that}\OperatorTok{,}\NormalTok{ you would need to comment out the }\BuiltInTok{Map}\OperatorTok{.}\AttributeTok{centerObject}\NormalTok{ command to keep the map }\ImportTok{from}\NormalTok{ moving to that location each time you run the script}\OperatorTok{.}

  
\NormalTok{Filter by Other Image Metadata        As first explained }\KeywordTok{in}\NormalTok{ Chap}\OperatorTok{.} \AttributeTok{F1}\OperatorTok{.}\DecValTok{3}\OperatorTok{,}\NormalTok{ the date and location }\KeywordTok{of}\NormalTok{ an image are characteristics stored }\ControlFlowTok{with}\NormalTok{ each image}\OperatorTok{.} \AttributeTok{Another}\NormalTok{ important factor }\KeywordTok{in}\NormalTok{ image processing is the cloud cover}\OperatorTok{,}\NormalTok{ an image}\OperatorTok{{-}}\NormalTok{level value computed }\ControlFlowTok{for}\NormalTok{ each image }\KeywordTok{in}\NormalTok{ many collections}\OperatorTok{,}\NormalTok{ including the Landsat and Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ collections}\OperatorTok{.} \AttributeTok{The}\NormalTok{ overall cloudiness score might be stored under different metadata tag names }\KeywordTok{in}\NormalTok{ different data sets}\OperatorTok{.} \AttributeTok{For}\NormalTok{ example}\OperatorTok{,} \ControlFlowTok{for}\NormalTok{ Sentinel}\OperatorTok{{-}}\DecValTok{2}\OperatorTok{,} \KeywordTok{this}\NormalTok{ overall cloudiness score is stored }\KeywordTok{in}\NormalTok{ the CLOUDY\_PIXEL\_PERCENTAGE metadata field}\OperatorTok{.} \AttributeTok{For}\NormalTok{ Landsat }\DecValTok{5}\OperatorTok{,}\NormalTok{ the ImageCollection we are using }\KeywordTok{in} \KeywordTok{this}\NormalTok{ example}\OperatorTok{,}\NormalTok{ the image}\OperatorTok{{-}}\NormalTok{level cloudiness score is stored using the tag CLOUD\_COVER}\OperatorTok{.} \AttributeTok{If}\NormalTok{ you are unfamiliar }\ControlFlowTok{with}\NormalTok{ how to find }\KeywordTok{this}\NormalTok{ information}\OperatorTok{,}\NormalTok{ these skills are first presented }\KeywordTok{in}\NormalTok{ Part F1}\OperatorTok{.}

\NormalTok{Here}\OperatorTok{,}\NormalTok{ we will access the ImageCollection that we just built using filterBounds and filterDate}\OperatorTok{,}\NormalTok{ and then further filter the images by the image}\OperatorTok{{-}}\NormalTok{level cloud cover score}\OperatorTok{,}\NormalTok{ using the filterMetadata }\KeywordTok{function}\OperatorTok{.}

\NormalTok{Next}\OperatorTok{,}\NormalTok{ let’s remove any images }\ControlFlowTok{with} \DecValTok{50}\OperatorTok{\%}\NormalTok{ or more cloudiness}\OperatorTok{.} \AttributeTok{As}\NormalTok{ will be described }\KeywordTok{in}\NormalTok{ subsequent chapters working }\ControlFlowTok{with}\NormalTok{ per}\OperatorTok{{-}}\NormalTok{pixel cloudiness information}\OperatorTok{,}\NormalTok{ you might want to retain those images }\KeywordTok{in}\NormalTok{ a real}\OperatorTok{{-}}\NormalTok{life study}\OperatorTok{,} \ControlFlowTok{if}\NormalTok{ you feel some values within cloudy images might be useful}\OperatorTok{.} \AttributeTok{For}\NormalTok{ now}\OperatorTok{,}\NormalTok{ to illustrate the filtering concept}\OperatorTok{,}\NormalTok{ let’s keep only images whose image}\OperatorTok{{-}}\NormalTok{level cloudiness values indicate that the cloud coverage is lower than }\DecValTok{50}\OperatorTok{\%.} \AttributeTok{Here}\OperatorTok{,}\NormalTok{ we will take the set already filtered by bounds and date}\OperatorTok{,}\NormalTok{ and further filter it using the cloud percentage into a }\KeywordTok{new}\NormalTok{ ImageCollection}\OperatorTok{.} \AttributeTok{Add} \KeywordTok{this}\NormalTok{ line to the script to filter by cloudiness and print the size to the }\BuiltInTok{Console}\OperatorTok{.}

\KeywordTok{var}\NormalTok{ L5FilteredLowCloudImages }\OperatorTok{=}\NormalTok{ imgColfilteredByDateHere  }
   \OperatorTok{.}\FunctionTok{filterMetadata}\NormalTok{(}\StringTok{\textquotesingle{}CLOUD\_COVER\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}less\_than\textquotesingle{}}\OperatorTok{,} \DecValTok{50}\NormalTok{)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{"Less than 50\% clouds in this area, 2000{-}2010"}\OperatorTok{,}  
\NormalTok{   L5FilteredLowCloudImages}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;} \CommentTok{// A smaller number}

\NormalTok{Filtering }\KeywordTok{in}\NormalTok{ an Efficient Order As you saw earlier }\KeywordTok{in}\NormalTok{ the hypothetical milk example}\OperatorTok{,}\NormalTok{ we typically filter}\OperatorTok{,}\NormalTok{ then map}\OperatorTok{,}\NormalTok{ and then reduce}\OperatorTok{,} \KeywordTok{in}\NormalTok{ that order}\OperatorTok{.} \AttributeTok{In}\NormalTok{ the same way that we would not want to call every store on Earth}\OperatorTok{,}\NormalTok{ preferring instead to narrow down the list }\KeywordTok{of}\NormalTok{ potential stores first}\OperatorTok{,}\NormalTok{ we filter images first }\KeywordTok{in}\NormalTok{ our workflow }\KeywordTok{in}\NormalTok{ Earth Engine}\OperatorTok{.} \AttributeTok{In}\NormalTok{ addition}\OperatorTok{,}\NormalTok{ you may have noticed that the ordering }\KeywordTok{of}\NormalTok{ the filters within the filtering stage also mattered }\KeywordTok{in}\NormalTok{ the milk example}\OperatorTok{.} \AttributeTok{This}\NormalTok{ is also }\KeywordTok{true} \KeywordTok{in}\NormalTok{ Earth Engine}\OperatorTok{.} \AttributeTok{For}\NormalTok{ problems }\ControlFlowTok{with}\NormalTok{ a non}\OperatorTok{{-}}\BuiltInTok{global}\NormalTok{ spatial component }\KeywordTok{in}\NormalTok{ which filterBounds is to be used}\OperatorTok{,}\NormalTok{ it is most efficient to }\ControlFlowTok{do}\NormalTok{ that spatial filtering first}\OperatorTok{.}

\NormalTok{In the code below}\OperatorTok{,}\NormalTok{ you will see that you can “chain” the filter commands}\OperatorTok{,}\NormalTok{ which are then executed }\ImportTok{from}\NormalTok{ left to right}\OperatorTok{.} \AttributeTok{Below}\OperatorTok{,}\NormalTok{ we chain the filters }\KeywordTok{in}\NormalTok{ the same order }\ImportTok{as}\NormalTok{ you specified above}\OperatorTok{.} \AttributeTok{Note}\NormalTok{ that it gives an ImageCollection }\KeywordTok{of}\NormalTok{ the same size }\ImportTok{as}\NormalTok{ when you applied the filters one at a time}\OperatorTok{.}

\KeywordTok{var}\NormalTok{ chainedFilteredSet }\OperatorTok{=}\NormalTok{ imgCol}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(}\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{())  }
   \OperatorTok{.}\FunctionTok{filterMetadata}\NormalTok{(}\StringTok{\textquotesingle{}CLOUD\_COVER\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}less\_than\textquotesingle{}}\OperatorTok{,} \DecValTok{50}\NormalTok{)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Chained: Less than 50\% clouds in this area, 2000{-}2010\textquotesingle{}}\OperatorTok{,}  
\NormalTok{   chainedFilteredSet}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the code below, we chain the filters in a more efficient order,
implementing filterBounds first. This, too, gives an ImageCollection of
the same size as when you applied the filters in the less efficient
order, whether the filters were chained or not.

var efficientFilteredSet = imgCol.filterBounds(Map.getCenter())\\
.filterDate(startDate, endDate)\\
.filterMetadata(`CLOUD\_COVER', `less\_than', 50);\\
print(`Efficient filtering: Less than 50\% clouds in this area,
2000-2010',\\
efficientFilteredSet.size());

Each of the two chained sets of operations will give the same result as
before for the number of images. While the second order is more
efficient, both approaches are likely to return the answer to the Code
Editor at roughly the same time for this very small example. The order
of operations is most important in larger problems in which you might be
challenged to manage memory carefully. As in the milk example in which
you narrowed geographically first, it is good practice in Earth Engine
to order the filters with the filterBounds first, followed by metadata
filters in order of decreasing specificity.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F40a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Now, with an efficiently filtered collection that satisfies our chosen
criteria, we will next explore the second stage: executing a function
for all of the images in the set.

\hypertarget{mapping-over-image-collections-in-earth-engine}{%
\subsection{Mapping over Image Collections in Earth
Engine}\label{mapping-over-image-collections-in-earth-engine}}

In Chap. F3.1, we calculated the Enhanced Vegetation Index (EVI) in very
small steps to illustrate band arithmetic on satellite images. In that
chapter, code was called once, on a single image. What if we wanted to
compute the EVI in the same way for every image of an entire
ImageCollection? Here, we use the key tool for the second part of the
workflow in Earth Engine, a .map command (Fig. F4.0.1). This is roughly
analogous to the step of making phone calls in the milk example that
began this chapter, in which you took a list of store names and
transformed it through effort into a list of milk prices.

Before beginning to code the EVI functionality, it's worth noting that
the word ``map'' is encountered in multiple settings during cloud-based
remote sensing, and it's important to be able to distinguish the uses. A
good way to think of it is that ``map'' can act as a verb or as a noun
in Earth Engine. There are two uses of ``map'' as a noun. We might refer
casually to ``the map,'' or more precisely to ``the Map panel''; these
terms refer to the place where the images are shown in the code
interface. A second way ``map'' is used as a noun is to refer to an
Earth Engine object, which has functions that can be called on it.
Examples of this are the familiar Map.addLayer and Map.setCenter. Where
that use of the word is intended, it will be shown in purple text and
capitalized in the Code Editor. What we are discussing here is the use
of .map as a verb, representing the idea of performing a set of actions
repeatedly on a set. This is typically referred to as ``mapping over the
set.''

To map a given set of operations efficiently over an entire
ImageCollection, the processing needs to be set up in a particular way.
Users familiar with other programming languages might expect to see
``loop'' code to do this, but the processing is not done exactly that
way in Earth Engine. Instead, we will create a function, and then map it
over the ImageCollection. To begin, envision creating a function that
takes exactly one parameter, an ee.Image. The function is then designed
to perform a specified set of operations on the input ee.Image and then,
importantly, returns an ee.Image as the last step of the function. When
we map that function over an ImageCollection, as we'll illustrate below,
the effect is that we begin with an ImageCollection, do operations to
each image, and receive a processed ImageCollection as the output.

What kinds of functions could we create? For example, you could imagine
a function taking an image and returning an image whose pixels have the
value 1 where the value of a given band was lower than a certain
threshold, and 0 otherwise. The effect of mapping this function would be
an entire ImageCollection of images with zeroes and ones representing
the results of that test on each image. Or you could imagine a function
computing a complex self-defined index and sending back an image of that
index calculated in each pixel. Here, we'll create a function to compute
the EVI for any input Landsat 5 image and return the one-band image for
which the index is computed for each pixel. Copy and paste the function
definition below into the Code Editor, adding it to the end of the
script from the previous section.

var makeLandsat5EVI = function(oneL5Image) \{ // compute the EVI for any
Landsat 5 image. Note it's specific to // Landsat 5 images due to the
band numbers. Don't run this exact // function for images from sensors
other than Landsat 5. // Extract the bands and divide by 1e4 to account
for scaling done. var nirScaled = oneL5Image.select(`SRvide(10000); var
redScaled = oneL5Image.select('SR\_B3').divide(10000); var blueScaled =
oneL5Image.select(`SR\_B1').divide(10000); // Calculate the numerator,
note that order goes from left to right. var numeratorEVI =
(nirScaled.subtract(redScaled)).multiply( 2.5); // Calculate the
denominator var denomClause1 = redScaled.multiply(6); var denomClause2 =
blueScaled.multiply(7.5); var denominatorEVI =
nirScaled.add(denomClause1).subtract(\\
denomClause2).add(1); // Calculate EVI and name it. var landsat5EVI =
numeratorEVI.divide(denominatorEVI).rename( `EVI'); return
(landsat5EVI);\\
\};

It is worth emphasizing that, in general, band names are specific to
each ImageCollection. As a result, if that function were run on an image
without the band `SR\_B4', for example, the function call would fail.
Here, we have emphasized in the function's name that it is specifically
for creating EVI for Landsat 5.

The function makeLandsat5EVI is built to receive a single image, select
the proper bands for calculating EVI, make the calculation, and return a
one-banded image. If we had the name of each image comprising our
ImageCollection, we could enter the names into the Code Editor and call
the function one at a time for each, assembling the images into
variables, and then combining them into an ImageCollection. This would
be very tedious and highly prone to mistakes: lists of items might get
mistyped, an image might be missed, etc. Instead, as mentioned above, we
will use .map. With the code below, let's print the information about
the cloud-filtered collection and display it, execute the .map command,
and explore the resulting ImageCollection.

var L5EVIimages = efficientFilteredSet.map(makeLandsat5EVI);\\
print(`Verifying that the .map gives back the same number of images:',\\
L5EVIimages.size());\\
print(L5EVIimages);

Map.addLayer(L5EVIimages, \{\}, `L5EVIimages', 1, 1);

After entering and executing this code, you will see a grayscale image.
If you look closely at the edges of the image, you might spot other
images drawn behind it in a way that looks somewhat like a stack of
papers on a table. This is the drawing of the ImageCollection made from
the makeLandsat5EVI function. You can select the Inspector panel and
click on one of the grayscale pixels to view the values of the entire
ImageCollection. After clicking on a pixel, look for the Series tag by
opening and closing the list of items. When you open that tag, you will
see a chart of the EVI values at that pixel, created by mapping the
makeLandsat5EVI function over the filtered ImageCollection.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F40b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{reducing-an-image-collection}{%
\subsection{Reducing an Image
Collection}\label{reducing-an-image-collection}}

The third part of the filter, map, reduce paradigm is ``reducing''
values in an ImageCollection to extract meaningful values (Fig. F4.0.1).
In the milk example, we reduced a large list of milk prices to find the
minimum value. The Earth Engine API provides a large set of reducers for
reducing a set of values to a summary statistic.

Here, you can think of each location, after the calculation of EVI has
been executed though the .map command, as having a list of EVI values on
it. Each pixel contains a potentially very large set of EVI values; the
stack might be 15 items high in one location and perhaps 200, 2000, or
200,000 items high in another location, especially if a looser set of
filters had been used.

The code below computes the mean value, at every pixel, of the
ImageCollection L5EVIimages created above. Add it at the bottom of your
code.

var L5EVImean = L5EVIimages.reduce(ee.Reducer.mean());\\
print(L5EVImean);\\
Map.addLayer(L5EVImean, \{\\
min: -1,\\
max: 2,\\
palette: {[}`red', `white', `green'{]}\\
\}, `Mean EVI');

Using the same principle, the code below computes and draws the median
value of the ImageCollection in every pixel.

var L5EVImedian = L5EVIimages.reduce(ee.Reducer.median());\\
print(L5EVImedian);\\
Map.addLayer(L5EVImedian, \{\\
min: -1,\\
max: 2,\\
palette: {[}`red', `white', `green'{]}\\
\}, `Median EVI');

\includegraphics{./F4/image43.png}

\begin{figure}

{\centering \includegraphics{./F4/image67.png}

}

\caption{Fig. 4.0.2 The effects of two reducers on mapped EVI values in
a filtered ImageCollection: mean image (above), and median image
(below)}

\end{figure}

There are many more reducers that work with an ImageCollection to
produce a wide range of summary statistics. Reducers are not limited to
returning only one item from the reduction. The minMax reducer, for
example, returns a two-band image for each band it is given, one for the
minimum and one for the maximum.

The reducers described here treat each pixel independently. In
subsequent chapters in Part F4, you will see other kinds of
reducers---for example, ones that summarize the characteristics in the
neighborhood surrounding each pixel.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F40c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-7}{%
\subsection*{Conclusion}\label{conclusion-7}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned about the paradigm of filter, map, reduce.
You learned how to use these tools to sift through, operate on, and
summarize a large set of images to suit your purposes. Using the Filter
functionality, you learned how to take a large ImageCollection and
filter away images that do not meet your criteria, retaining only those
images that match a given set of characteristics. Using the Map
functionality, you learned how to apply a function to each image in an
ImageCollection, treating each image one at a time and executing a
requested set of operations on each. Using the Reduce functionality, you
learned how to summarize the elements of an ImageCollection, extracting
summary values of interest. In the subsequent chapters of Part 4, you
will encounter these concepts repeatedly, manipulating image collections
according to your project needs using the building blocks seen here. By
building on what you have done in this chapter, you will grow in your
ability to do sophisticated projects in Earth Engine.

\hypertarget{exploring-image-collections}{%
\section{Exploring Image
Collections}\label{exploring-image-collections}}

:::\{.callout-tip\} \#\# Chapter Information

\hypertarget{author-6}{%
\subsubsection*{Author}\label{author-6}}

Gennadii Donchyts

\hypertarget{overview-8}{%
\subsubsection*{Overview}\label{overview-8}}

This chapter teaches how to explore image collections, including their
spatiotemporal extent, resolution, and values stored in images and image
properties. You will learn how to map and inspect image collections
using maps, charts, and interactive tools, and how to compute different
statistics of values stored in image collections using reducers.

\hypertarget{learning-outcomes-8}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-8}}

\begin{itemize}
\tightlist
\item
  Inspecting the spatiotemporal extent and resolution of image
  collections by mapping image geometry and plotting image time
  properties.
\item
  Exploring properties of images stored in an ImageCollection by
  plotting charts and deriving statistics.
\item
  Filtering image collections by using stored or computed image
  properties.
\item
  Exploring the distribution of values stored in image pixels of an
  ImageCollection through percentile reducers.
\end{itemize}

\hypertarget{assumes-you-know-how-to-8}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-8}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\item
  Summarize an ImageCollection with reducers (Chap. F4.0).
\end{itemize}

\begin{center}\rule{0.5\linewidth}{0.5pt}\end{center}

In the previous chapter (Chap. F4.0), the filter, map, reduce paradigm
was introduced. The main goal of this chapter is to demonstrate some of
the ways that those concepts can be used within Earth Engine to better
understand the variability of values stored in image collections. Sect.
1 demonstrates how time-dependent values stored in the images of an
ImageCollection can be inspected using the Code Editor user interface
after filtering them to a limited spatiotemporal range (i.e., geometry
and time ranges). Sect. 2 shows how the extent of images, as well as
basic statistics, such as the number of observations, can be visualized
to better understand the spatiotemporal extent of image collections.
Then, Sects. 3 and 4 demonstrate how simple reducers such as mean and
median, and more advanced reducers such as percentiles, can be used to
better understand how the values of a filtered ImageCollection are
distributed.

\hypertarget{filtering-and-inspecting-an-image-collection}{%
\subsection{Filtering and Inspecting an Image
Collection}\label{filtering-and-inspecting-an-image-collection}}

We will focus on the area in and surrounding Lisbon, Portugal. Below, we
will define a point, lisbonPoint, located in the city; access the very
large Landsat ImageCollection and limit it to the year 2020 and to the
images that contain Lisbon; and select bands 6, 5, and 4 from each of
the images in the resulting filtered ImageCollection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define a region of interest as a point in Lisbon, Portugal.  }
\KeywordTok{var}\NormalTok{ lisbonPoint }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{(}\OperatorTok{{-}}\FloatTok{9.179473}\OperatorTok{,} \FloatTok{38.763948}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Center the map at that point.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(lisbonPoint}\OperatorTok{,} \DecValTok{16}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// filter the large ImageCollection to be just images from 2020  }
\CommentTok{// around Lisbon. From each image, select true{-}color bands to draw  }
\KeywordTok{var}\NormalTok{ filteredIC }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_TOA\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(lisbonPoint)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}B6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Add the filtered ImageCollection so that we can inspect values  }
\CommentTok{// via the Inspector tool  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(filteredIC}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}TOA image collection\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The three selected bands (which correspond to SWIR1, NIR, and Red)
display a false-color image that accentuates differences between
different land covers (e.g., concrete, vegetation) in Lisbon. With the
Inspector tab highlighted (Fig. F4.1.1), clicking on a point will bring
up the values of bands 6, 5, and 4 from each of the images. If you open
the Series option, you'll see the values through time. For the specified
point and for all other points in Lisbon (since they are all enclosed in
the same Landsat scene), there are 16 images gathered in 2020. By
following one of the graphed lines (in blue, yellow, or red) with your
finger, you should be able to count that many distinct values. Moving
the mouse along the lines will show the specific values and the image
dates.

\begin{figure}

{\centering \includegraphics{./F4/image22.png}

}

\caption{Fig. F4.1.1 Inspect values in an ImageCollection at a selected
point by making use of the Inspector tool in the Code Editor}

\end{figure}

We can also show this kind of chart automatically by making use of the
ui.Chart function of the Earth Engine API. The following code snippet
should result in the same chart as we could observe in the Inspector
tab, assuming the same pixel is clicked.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Construct a chart using values queried from image collection.  }
\KeywordTok{var}\NormalTok{ chart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(\{  }
   \DataTypeTok{imageCollection}\OperatorTok{:}\NormalTok{ filteredIC}\OperatorTok{,}  
   \DataTypeTok{region}\OperatorTok{:}\NormalTok{ lisbonPoint}\OperatorTok{,}  
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:} \DecValTok{10}  
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Show the chart in the Console.  }
\FunctionTok{print}\NormalTok{(chart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F41a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{how-many-images-are-there-everywhere-on-earth}{%
\subsection{How Many Images Are There, Everywhere on
Earth?}\label{how-many-images-are-there-everywhere-on-earth}}

Suppose we are interested to find out how many valid observations we
have at every map pixel on Earth for a given ImageCollection. This
enormously computationally demanding task is surprisingly easy to do in
Earth Engine. The API provides a set of reducer functions to summarize
values to a single number in each pixel, as described in Chap. F4.0. We
can apply this reducer, count, to our filtered ImageCollection with the
code below. We'll return to the same data set and filter for 2020, but
without the geographic limitation. This will assemble images from all
over the world, and then count the number of images in each pixel. The
following code does that count, and adds the resulting image to the map
with a predefined red/yellow/green color palette stretched between
values 0 and 50. Continue pasting the code below into the same script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// compute and show the number of observations in an image collection  }
\KeywordTok{var}\NormalTok{ count }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_TOA\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}B6\textquotesingle{}}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{count}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// add white background and switch to HYBRID basemap  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}white\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \KeywordTok{true}\OperatorTok{,} \FloatTok{0.5}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{\textquotesingle{}HYBRID\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// show image count  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(count}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{50}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}d7191c\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}fdae61\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ffffbf\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}a6d96a\textquotesingle{}}\OperatorTok{,}       \StringTok{\textquotesingle{}1a9641\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}landsat 8 image count (2020)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Center the map at that point.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(lisbonPoint}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Run the command and zoom out. If the count of images over the entire
Earth is viewed, the resulting map should look like Fig. F4.1.2. The
created map data may take a few minutes to fully load in.

\begin{figure}

{\centering \includegraphics{./F4/image27.png}

}

\caption{Fig. F4.1.2 The number of Landsat 8 images acquired during
2020}

\end{figure}

Note the checkered pattern, somewhat reminiscent of a Mondrian painting.
To understand why the image looks this way, it is useful to consider the
overlapping image footprints. As Landsat passes over, each image is wide
enough to produce substantial ``sidelap'' with the images from the
adjacent paths, which are collected at different dates according to the
satellite's orbit schedule. In the north-south direction, there is also
some overlap to ensure that there are no gaps in the data. Because these
are served as distinct images and stored distinctly in Earth Engine, you
will find that there can be two images from the same day with the same
value for points in these overlap areas. Depending on the purposes of a
study, you might find a way to ignore the duplicate pixel values during
the analysis process.

You might have noticed that we summarized a single band from the
original ImageCollection to ensure that the resulting image would give a
single count in each pixel. The count reducer operates on every band
passed to it. Since every image has the same number of bands, passing an
ImageCollection of all seven Landsat bands to the count reducer would
have returned seven identical values of 16 for every point. To limit any
confusion from seeing the same number seven times, we selected one of
the bands from each image in the collection. In your own work, you might
want to use a different reducer, such as a median operation, that would
give different, useful answers for each band. A few of these reducers
are described below.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F41b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{reducing-image-collections-to-understand-band-values}{%
\subsection{Reducing Image Collections to Understand Band
Values}\label{reducing-image-collections-to-understand-band-values}}

As we have seen, you could click at any point on Earth's surface and see
both the number of Landsat images recorded there in 2020 and the values
of any image in any band through time. This is impressive and perhaps
mind-bending, given the enormous amount of data in play. In this section
and the next, we will explore two ways to summarize the numerical values
of the bands---one straightforward way and one more complex but highly
powerful way to see what information is contained in image collections.

First, we will make a new layer that represents the mean value of each
band in every pixel across every image from 2020 for the filtered set,
add this layer to the layer set, and explore again with the Inspector.
The previous section's count reducer was called directly using a sort of
simple shorthand; that could be done similarly here by calling mean on
the assembled bands. In this example, we will use the reducer to get the
mean using the more general reduce call. Continue pasting the code below
into the same script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Zoom to an informative scale for the code that follows.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(lisbonPoint}\OperatorTok{,} \DecValTok{10}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add a mean composite image.  }
\KeywordTok{var}\NormalTok{ meanFilteredIC }\OperatorTok{=}\NormalTok{ filteredIC}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mean}\NormalTok{())}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(meanFilteredIC}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}   \StringTok{\textquotesingle{}Mean values within image collection\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now, let's look at the median value for each band among all the values
gathered in 2020. Using the code below, calculate the median and explore
the image with the Inspector. Compare this image briefly to the mean
image by eye and by clicking in a few pixels in the Inspector. They
should have different values, but in most places they will look very
similar.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Add a median composite image.  }
\KeywordTok{var}\NormalTok{ medianFilteredIC }\OperatorTok{=}\NormalTok{ filteredIC}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{median}\NormalTok{())}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(medianFilteredIC}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}   \StringTok{\textquotesingle{}Median values within image collection\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

There is a wide range of reducers available in Earth Engine. If you are
curious about which reducers can be used to summarize band values across
a collection of images, use the Docs tab in the Code Editor to list all
reducers and look for those beginning with ee.Reducer.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F41c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{compute-multiple-percentile-images-for-an-image-collection}{%
\subsection{Compute Multiple Percentile Images for an Image
Collection}\label{compute-multiple-percentile-images-for-an-image-collection}}

One particularly useful reducer that can help you better understand the
variability of values in image collections is ee.Reducer.percentile. The
nth percentile gives the value that is the nth largest in a set. In this
context, you can imagine accessing all of the values for a given band in
a given ImageCollection for a given pixel and sorting them. The 30th
percentile, for example, is the value 30\% of the way along the list
from smallest to largest. This provides an easy way to explore the
variability of the values in image collections by computing a cumulative
density function of values on a per-pixel basis. The following code
shows how we can calculate a single 30th percentile on a per-pixel and
per-band basis for our Landsat 8 ImageCollection. Continue pasting the
code below into the same script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// compute a single 30\% percentile  }
\KeywordTok{var}\NormalTok{ p30 }\OperatorTok{=}\NormalTok{ filteredIC}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{percentile}\NormalTok{([}\DecValTok{30}\NormalTok{]))}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(p30}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \FloatTok{0.05}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.35}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}30\%\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image74.png}

}

\caption{Fig. F4.1.3 Landsat 8 TOA reflectance 30th percentile image
computed for ImageCollection with images acquired during 2020}

\end{figure}

We can see that the resulting composite image (Fig. 4.1.3) has almost no
cloudy pixels present for this area. This happens because cloudy pixels
usually have higher reflectance values. At the lowest end of the values,
other unwanted effects like cloud or hill shadows typically have very
low reflectance values. This is why this 30th percentile composite image
looks so much cleaner than the mean composite image (meanFilteredIC)
calculated earlier. Note that the reducers operate per pixel: adjacent
pixels are drawn from different images. This means that one pixel's
value could be taken from an image from one date, and the adjacent
pixel's value drawn from an entirely different period. Although, like
the mean and median images, percentile images such as that seen in Fig.
F4.1.3 never existed on a single day, composite images allow us to view
Earth's surface without the noise that can make analysis difficult.

We can explore the range of values in an entire ImageCollection by
viewing a series of increasingly bright percentile images, as shown in
Fig. F4.1.4. Paste and run the following code.

var percentiles = {[}0, 10, 20, 30, 40, 50, 60, 70, 80{]};

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// let\textquotesingle{}s compute percentile images and add them as separate layers  }
\NormalTok{percentiles}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(p) \{   }\KeywordTok{var}\NormalTok{ image }\OperatorTok{=}\NormalTok{ filteredIC}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{percentile}\NormalTok{([p]))}\OperatorTok{;}   \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(image}\OperatorTok{,}\NormalTok{ \{  }
       \DataTypeTok{min}\OperatorTok{:} \FloatTok{0.05}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.35}\NormalTok{   \}}\OperatorTok{,}\NormalTok{ p }\OperatorTok{+} \StringTok{\textquotesingle{}\%\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that the code adds every percentile image as a separate map layer,
so you need to go to the Layers control and show/hide different layers
to explore differences. Here, we can see that low-percentile composite
images depict darker, low-reflectance land features, such as water and
cloud or hill shadows, while higher-percentile composite images
(\textgreater70\% in our example) depict clouds and any other
atmospheric or land effects corresponding to bright reflectance values.

\begin{figure}

{\centering \includegraphics{./F4/image55.png}

}

\caption{Fig. F4.1.4 Landsat 8 TOA reflectance percentile composite
images}

\end{figure}

Earth Engine provides a very rich API, allowing users to explore image
collections to better understand the extent and variability of data in
space, time, and across bands, as well as tools to analyze values stored
in image collections in a frequency domain. Exploring these values in
different forms should be the first step of any study before developing
data analysis algorithms.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F41d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-8}{%
\subsection*{Conclusion}\label{conclusion-8}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you have learned different ways to explore image
collections using Earth Engine in addition to looking at individual
images. You have learned that image collections in Earth Engine may have
global footprints as well as images with a smaller, local footprint, and
how to visualize the number of images in a given filtered
ImageCollection. You have learned how to explore the temporal and
spatial extent of images stored in image collections, and how to quickly
examine the variability of values in these image collections by
computing simple statistics like mean or median, as well as how to use a
percentile reducer to better understand this variability.

\hypertarget{references-4}{%
\subsection*{References}\label{references-4}}
\addcontentsline{toc}{subsection}{References}

Wilson AM, Jetz W (2016) Remotely sensed high-resolution global cloud
dynamics for predicting ecosystem and biodiversity distributions. PLoS
Biol 14:e1002415. https://doi.org/10.1371/journal.pbio.1002415

\hypertarget{aggregating-images-for-time-series}{%
\section{Aggregating Images for Time
Series}\label{aggregating-images-for-time-series}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-7}{%
\subsubsection*{Author}\label{author-7}}

Ujaval Gandhi

\hypertarget{overview-9}{%
\subsubsection*{Overview}\label{overview-9}}

Many remote sensing datasets consist of repeated observations over time.
The interval between observations can vary widely. The Global
Precipitation Measurement dataset, for example, produces observations of
rain and snow worldwide every three hours. The Climate Hazards Group
InfraRed Precipitation with Station (CHIRPS) project produces a gridded
global dataset at the daily level and also for each five-day period. The
Landsat 8 mission produces a new scene of each location on Earth every
16 days. With its constellation of two satellites, the Sentinel-2
mission images every location every five days.

Many applications, however, require computing aggregations of data at
time intervals different from those at which the datasets were produced.
For example, for determining rainfall anomalies, it is useful to compare
monthly rainfall against a long-period monthly average.

While individual scenes are informative, many days are cloudy, and it is
useful to build a robust cloud-free time series for many applications.
Producing less cloudy or even cloud-free composites can be done by
aggregating data to form monthly, seasonal, or yearly composites built
from individual scenes. For example, if you are interested in detecting
long-term changes in an urban landscape, creating yearly median
composites can enable you to detect change patterns across long time
intervals with less worry about day-to-day noise.

This chapter will cover the techniques for aggregating individual images
from a time series at a chosen interval. We will take the CHIRPS time
series of rainfall for one year and aggregate it to create a monthly
rainfall time series.

\hypertarget{learning-outcomes-9}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-9}}

\begin{itemize}
\tightlist
\item
  Using the Earth Engine API to work with dates.
\item
  Aggregating values from an ImageCollection to calculate monthly,
  seasonal, or yearly images.
\item
  Plotting the aggregated time series at a given location.
\end{itemize}

\hypertarget{assumes-you-know-how-to-9}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-9}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  \hspace{0pt}\hspace{0pt}Create a graph using ui.Chart (Chap. F1.3).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Summarize an ImageCollection with reducers (Chap. F4.0, Chap. F4.1).
\item
  Inspect an Image and an ImageCollection, as well as their properties
  (Chap. F4.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-7}{%
\subsection*{Introduction}\label{introduction-7}}

CHIRPS is a high-resolution global gridded rainfall dataset that
combines satellite-measured precipitation with ground station data in a
consistent, long time-series dataset. The data are provided by the
University of California, Santa Barbara, and are available from 1981 to
the present. This dataset is extremely useful in drought monitoring and
assessing global environmental change over land. The satellite data are
calibrated with ground station observations to create the final product.

In this exercise, we will work with the CHIRPS dataset using the pentad.
A pentad represents the grouping of five days. There are six pentads in
a calendar month, with five pentads of exactly five days each and one
pentad with the remaining three to six days of the month. Pentads reset
at the beginning of each month, and the first day of every month is the
start of a new pentad. Values at a given pixel in the CHIRPS dataset
represent the total precipitation in millimeters over the pentad.

\hypertarget{filtering-an-image-collection}{%
\subsection{Filtering an Image
Collection}\label{filtering-an-image-collection}}

We will start by accessing the CHIRPS Pentad collection and filtering it
to create a time series for a single year.

var chirps = ee.ImageCollection(`UCSB-CHG/CHIRPS/PENTAD');\\
var startDate = `2019-01-01';\\
var endDate = `2020-01-01';\\
var yearFiltered = chirps.filter(ee.Filter.date(startDate, endDate));

print(yearFiltered, `Date-filtered CHIRPS images');

The CHIRPS collection contains one image for every pentad. The filtered
collection above is filtered to contain one year, which equates to 72
global images. If you expand the printed collection in the Console, you
will be able to see the metadata for individual images; note that their
date stamps indicate that they are spaced evenly every five days (Fig.
F4.2.1).

\begin{figure}

{\centering \includegraphics{./F4/image25.png}

}

\caption{Fig. F4.2.1 CHIRPS time series for one year}

\end{figure}

Each image's pixel values store the total precipitation during the
pentad. Without aggregation to a period that matches other datasets,
these layers are not very useful. For hydrological analysis, we
typically need the total precipitation for each month or for a season.
Let's aggregate this collection so that we have 12 images---one image
per month, with pixel values that represent the total precipitation for
that month.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F42a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{working-with-dates}{%
\subsection{Working with Dates}\label{working-with-dates}}

To aggregate the time series, we need to learn how to create and
manipulate dates programmatically. This section covers some functions
from the ee.Date module that will be useful.

The Earth Engine API has a function called ee.Date.fromYMD that is
designed to create a date object from year, month, and day values. The
following code snippet shows how to define a variable containing the
year value and create a date object from it. Paste the following code in
a new script:

var chirps = ee.ImageCollection(`UCSB-CHG/CHIRPS/PENTAD');\\
var year = 2019;\\
var startDate = ee.Date.fromYMD(year, 1, 1);

Now, let's determine how to create an end date in order to be able to
specify a desired time interval. The preferred way to create a date
relative to another date is using the advance function. It takes two
parameters---a delta value and the unit of time---and returns a new
date. The code below shows how to create a date one year in the future
from a given date. Paste it into your script.

var endDate = startDate.advance(1, `year');

Next, paste the code below to perform filtering of the CHIRPS data using
these calculated dates. After running it, check that you had accurately
set the dates by looking for the dates of the images inside the printed
result..

var yearFiltered = chirps\\
.filter(ee.Filter.date(startDate, endDate));\\
print(yearFiltered, `Date-filtered CHIRPS images');

Another date function that is very commonly used across Earth Engine is
millis. This function takes a date object and returns the number of
milliseconds since the arbitrary reference date of the start of the year
1970: 1970-01-01T00:00:00Z. This is known as the ``Unix Timestamp''; it
is a standard way to convert dates to numbers and allows for easy
comparison between dates with high precision. Earth Engine objects store
the timestamps for images and features in special properties called
system:time\_start and system:time\_end. Both of these properties need
to be supplied with a number instead of dates, and the millis function
can help you do that. You can print the result of calling this function
and check for yourself.

print(startDate, `Start date');\\
print(endDate, `End date');

print(`Start date as timestamp', startDate.millis());\\
print(`End date as timestamp', endDate.millis());

We will use the millis function in the next section when we need to set
the system:time\_start and system:time\_end properties of the aggregated
images.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F42b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{aggregating-images}{%
\subsection{Aggregating Images}\label{aggregating-images}}

Now we can start aggregating the pentads into monthly sums. The process
of aggregation has two fundamental steps. The first is to determine the
beginning and ending dates of one time interval (in this case, one
month), and the second is to sum up all of the values (in this case, the
pentads) that fall within each interval. To begin, we can envision that
the resulting series will contain 12 images. To prepare to create an
image for each month, we create an ee.List of values from 1 to 12. We
can use the ee.List.sequence function, as first presented in Chap. F1.0,
to create the list of items of type ee.Number. Continuing with the
script of the previous section, paste the following code:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Aggregate this time series to compute monthly images.  }
\CommentTok{// Create a list of months  }
\KeywordTok{var}\NormalTok{ months }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{List}\OperatorTok{.}\FunctionTok{sequence}\NormalTok{(}\DecValTok{1}\OperatorTok{,} \DecValTok{12}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, we write a function that takes a single month as the input and
returns an aggregated image for that month. Given beginningMonth as an
input parameter, we first create a start and end date for that month
based on the year and month variables. Then we filter the collection to
find all images for that month. To create a monthly precipitation image,
we apply ee.Reducer.sum to reduce the six pentad images for a month to a
single image holding the summed value across the pentads. We also
expressly set the timestamp properties system:time\_start and
system:time\_end of the resulting summed image. We can also set year and
month, which will help us filter the resulting collection later.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Write a function that takes a month number  }
\CommentTok{// and returns a monthly image.  }
\KeywordTok{var}\NormalTok{ createMonthlyImage }\OperatorTok{=} \KeywordTok{function}\NormalTok{(beginningMonth) \{   }\KeywordTok{var}\NormalTok{ startDate }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Date}\OperatorTok{.}\FunctionTok{fromYMD}\NormalTok{(year}\OperatorTok{,}\NormalTok{ beginningMonth}\OperatorTok{,} \DecValTok{1}\NormalTok{)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=}\NormalTok{ startDate}\OperatorTok{.}\FunctionTok{advance}\NormalTok{(}\DecValTok{1}\OperatorTok{,} \StringTok{\textquotesingle{}month\textquotesingle{}}\NormalTok{)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ monthFiltered }\OperatorTok{=}\NormalTok{ yearFiltered  }
       \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{date}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate))}\OperatorTok{;}   \CommentTok{// Calculate total precipitation.   var total = monthFiltered.reduce(ee.Reducer.sum());   return total.set(\{       \textquotesingle{}system:time\_start\textquotesingle{}: startDate.millis(),       \textquotesingle{}system:time\_end\textquotesingle{}: endDate.millis(),       \textquotesingle{}year\textquotesingle{}: year,       \textquotesingle{}month\textquotesingle{}: beginningMonth   \});  }
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We now have an ee.List containing items of type ee.Number from 1 to 12,
with a function that can compute a monthly aggregated image for each
month number. All that is left to do is to map the function over the
list. As described in Chaps. F4.0 and F4.1, the map function passes over
each image in the list and runs createMonthlyImage. The function first
receives the number ``1'' and executes, returning an image to Earth
Engine. Then it runs on the number ``2'', and so on for all 12 numbers.
The result is a list of monthly images for each month of the year.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// map() the function on the list of months  }
\CommentTok{// This creates a list with images for each month in the list  }
\KeywordTok{var}\NormalTok{ monthlyImages }\OperatorTok{=}\NormalTok{ months}\OperatorTok{.}\FunctionTok{map}\NormalTok{(createMonthlyImage)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We can create an ImageCollection from this ee.List of images using the
ee.ImageCollection.fromImages function.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create an ee.ImageCollection.  }
\KeywordTok{var}\NormalTok{ monthlyCollection }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{ImageCollection}\OperatorTok{.}\FunctionTok{fromImages}\NormalTok{(monthlyImages)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(monthlyCollection)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We have now successfully computed an aggregated collection from the
source ImageCollection by filtering, mapping, and reducing, as described
in Chaps. F4.0 and F4.1. Expand the printed collection in the Console
and you can verify that we now have 12 images in the newly created
ImageCollection (Fig. F4.2.2).

\begin{figure}

{\centering \includegraphics{./F4/image90.png}

}

\caption{Fig. F4.2.2 Aggregated time series}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F42c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{plotting-time-series}{%
\subsection{Plotting Time Series}\label{plotting-time-series}}

One useful application of gridded precipitation datasets is to analyze
rainfall patterns. We can plot a time-series chart for a location using
the newly computed time series. We can plot the pixel value at any given
point or polygon. Here we create a point geometry for a given
coordinate. Continuing with the script of the previous section, paste
the following code:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a point with coordinates for the city of Bengaluru, India.  }
\KeywordTok{var}\NormalTok{ point }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{(}\FloatTok{77.5946}\OperatorTok{,} \FloatTok{12.9716}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Earth Engine comes with a built-in ui.Chart.image.series function that
can plot time series. In addition to the imageCollection and region
parameters, we need to supply a scale value. The CHIRPS data catalog
page indicates that the resolution of the data is
\hspace{0pt}\hspace{0pt}5566 meters, so we can use that as the scale.
The resulting chart is printed in the Console.

var chart = ui.Chart.image.series(\{\\
imageCollection: monthlyCollection,\\
region: point,\\
reducer: ee.Reducer.mean(),\\
scale: 5566,\\
\});\\
print(chart);

We can make the chart more informative by adding axis labels and a
title. The setOptions function allows us to customize the chart using
parameters from Google Charts. To customize the chart, paste the code
below at the bottom of your script. The effect will be to see two charts
in the editor: one with the old view of the data, and one with the
customized chart.

var chart = ui.Chart.image.series(\{\\
imageCollection: monthlyCollection,\\
region: point,\\
reducer: ee.Reducer.mean(),\\
scale: 5566\\
\}).setOptions(\{\\
lineWidth: 1,\\
pointSize: 3,\\
title: `Monthly Rainfall at Bengaluru',\\
vAxis: \{\\
title: `Rainfall (mm)' \},\\
hAxis: \{\\
title: `Month',\\
gridlines: \{\\
count: 12 \}\\
\}\\
\});print(chart);

The customized chart (Fig. F4.2.3) shows the typical rainfall pattern in
the city of Bengaluru, India. Bengaluru has a temperate climate, with
pre-monsoon rains in April and May cooling down the city and a moderate
monsoon season lasting from June to September.

\begin{figure}

{\centering \includegraphics{./F4/image32.png}

}

\caption{Fig. F4.2.3 Monthly rainfall chart}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F42d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-9}{%
\subsection*{Conclusion}\label{conclusion-9}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to aggregate a collection to months and
plot the resulting time series for a location. This chapter also
introduced useful functions for working with the dates that will be used
across many different applications. You also learned how to iterate over
a list using the map function. The technique of mapping a function over
a list or collection is essential for processing data. Mastering this
technique will allow you to scale your analysis using the parallel
computing capabilities of Earth Engine.

\hypertarget{references-5}{%
\subsection*{References}\label{references-5}}
\addcontentsline{toc}{subsection}{References}

Banerjee A, Chen R, Meadows ME, et al (2020) An analysis of long-term
rainfall trends and variability in the Uttarakhand Himalaya using Google
Earth Engine. Remote Sens 12:709. https://doi.org/10.3390/rs12040709

Funk C, Peterson P, Landsfeld M, et al (2015) The climate hazards
infrared precipitation with stations -- a new environmental record for
monitoring extremes. Sci Data 2:1--21.
https://doi.org/10.1038/sdata.2015.66

Okamoto K, Ushio T, Iguchi T, et al (2005) The global satellite mapping
of precipitation (GSMaP) project. In: International Geoscience and
Remote Sensing Symposium (IGARSS). pp 3414--3416

\hypertarget{clouds-and-image-compositing}{%
\section{Clouds and Image
Compositing}\label{clouds-and-image-compositing}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-8}{%
\subsubsection*{Author}\label{author-8}}

:

Txomin Hermosilla, Saverio Francini, Andréa P. Nicolau, Michael A.
Wulder, Joanne C. White, Nicholas C. Coops, Gherardo Chirici

\hypertarget{overview-10}{%
\subsubsection*{Overview}\label{overview-10}}

The purpose of this chapter is to provide necessary context and
demonstrate different approaches for image composite generation when
using data quality flags, using an initial example of removing cloud
cover. We will examine different filtering options, demonstrate an
approach for cloud masking, and provide additional opportunities for
image composite development. Pixel selection for composite development
can exclude unwanted pixels---such as those impacted by cloud, shadow,
and smoke or haze---and can also preferentially select pixels based upon
proximity to a target date or a preferred sensor type.

\hypertarget{learning-outcomes-10}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-10}}

\begin{itemize}
\tightlist
\item
  Understanding and applying satellite-specific cloud mask functions.
\item
  Incorporating images from different sensors.
\item
  Using focal functions to fill in data gaps.
\end{itemize}

\hypertarget{assumes-you-know-how-to-10}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-10}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\item
  Use band scaling factors (Chap. F3.1).
\item
  Perform pixel-based transformations (Chap. F3.1).
\item
  Use neighborhood-based image transformations (Chap. F3.2).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Summarize an ImageCollection with reducers (Chap. F4.0, Chap. F4.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-8}{%
\subsection*{Introduction}\label{introduction-8}}

In many respects, satellite remote sensing is an ideal source of data
for monitoring large or remote regions. However, cloud cover is one of
the most common limitations of optical sensors in providing continuous
time series of data for surface mapping and monitoring. This is
particularly relevant in tropical, polar, mountainous, and high-latitude
areas, where clouds are often present. Many studies have addressed the
extent to which cloudiness can restrict the monitoring of various
regions (Zhu and Woodcock 2012, 2014; Eberhardt et al.~2016; Martins et
al.~2018).

Clouds and cloud shadows reduce the view of optical sensors and
completely block or obscure the spectral response from Earth's surface
(Cao et al.~2020). Working with pixels that are cloud-contaminated can
significantly influence the accuracy and information content of products
derived from a variety of remote sensing activities, including land
cover classification, vegetation modeling, and especially change
detection, where unscreened clouds might be mapped as false changes
(Braaten et al.~2015, Zhu et al.~2015). Thus, the information provided
by cloud detection algorithms is critical to exclude clouds and cloud
shadows from subsequent processing steps.

Historically, cloud detection algorithms derived the cloud information
by considering a single date-image and sun illumination geometry (Irish
et al.~2006, Huang et al.~2010). In contrast, current, more accurate
cloud detection algorithms are based on the analysis of Landsat time
series (Zhu and Woodcock 2014, Zhu and Helmer 2018). Cloud detection
algorithms inform on the presence of clouds, cloud shadows, and other
atmospheric conditions (e.g., presence of snow). The presence and extent
of cloud contamination within a pixel is currently provided with Landsat
and Sentinel-2 imagery as ancillary data via quality flags at the pixel
level. Additionally, quality flags also inform on other
acquisition-related conditions, including radiometric saturation and
terrain occlusion, which enables us to assess the usefulness and
convenience of inclusion of each pixel in subsequent analyses. The
quality flags are ideally suited to reduce users' manual supervision and
maximize the automatic processing approaches.

Most automated algorithms (for classification or change detection, for
example) work best on images free of clouds and cloud shadows, that
cover the full area without spatial or spectral inconsistencies. Thus,
the image representation over the study area should be seamless,
containing as few data gaps as possible. Image compositing techniques
are primarily used to reduce the impact of clouds and cloud shadows, as
well as aerosol contamination, view angle effects, and data volumes
(White et al.~2014). Compositing approaches typically rely on the
outputs of cloud detection algorithms and quality flags to include or
exclude pixels from the resulting composite products (Roy et al.~2010).
Epochal image composites help overcome the limited availability of
cloud-free imagery in some areas, and are constructed by considering the
pixels from all images acquired in a given period (e.g., season, year).

The information provided by the cloud masks and pixel flags guides the
establishment of rules to rank the quality of the pixels based on the
presence of and distance to clouds, cloud shadows, or atmospheric haze
(Griffiths et al.~2010). Higher scores are assigned to pixels with more
desirable conditions, based on the presence of clouds and also other
acquisition circumstances, such as acquisition date or sensor. Those
pixels with the highest scores are included in the subsequent composite
development. Image compositing approaches enable users to define the
rules that are most appropriate for their particular information needs
and study area to generate imagery covering large areas instead of being
limited to the analysis of single scenes (Hermosilla et al.~2015,
Loveland and Dwyer 2012). Moreover, generating image composites at
regular intervals (e.g., annually) allows for the analysis of long
temporal series over large areas, fulfilling a critical information need
for monitoring programs.

The general workflow to generate a cloud-free composite involves:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Defining your area of interest (AOI).
\item
  Filtering (ee.Filter) the satellite ImageCollection to desired
  parameters.
\item
  Applying a cloud mask.
\item
  Reducing (ee.Reducer) the collection to generate a composite.
\item
  Using the GEE-BAP application to generate annual best-available-pixel
  image composites by globally combining multiple Landsat sensors and
  images.
\end{enumerate}

Additional steps may be necessary to improve the composite generated.
These steps will be explained in the following sections.

\hypertarget{cloud-filter-and-cloud-mask}{%
\subsection{Cloud Filter and Cloud
Mask}\label{cloud-filter-and-cloud-mask}}

The first step is to define your AOI and center the map. The goal is to
create a nationwide composite for the country of Colombia. We will use
the Large Scale International Boundary (2017) simplified dataset from
the US Department of State (USDOS), which contains polygons for all
countries of the world.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// {-}{-}{-}{-}{-}{-}{-}{-}{-}{-} Section 1 {-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}  }
  
\CommentTok{// Define the AOI.  }
\KeywordTok{var}\NormalTok{ country }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}USDOS/LSIB\_SIMPLE/2017\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{equals}\NormalTok{(}\StringTok{\textquotesingle{}country\_na\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Colombia\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Center the Map. The second parameter is zoom level.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(country}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We will start creating a composite from the Landsat 8 collection. First,
we define two time variables: startDate and endDate. Here, we will
create a composite for the year 2019. Then, we will define a collection
for the Landsat 8 Level 2, Collection 2, Tier 1 variable and filter it
to our AOI and time period. We define and use a function to apply
scaling factors to the Landsat 8 Collection 2 data.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define time variables.  }
\KeywordTok{var}\NormalTok{ startDate }\OperatorTok{=} \StringTok{\textquotesingle{}2019{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=} \StringTok{\textquotesingle{}2019{-}12{-}31\textquotesingle{}}\OperatorTok{;}  
  
\CommentTok{// Load and filter the Landsat 8 collection.  }
\KeywordTok{var}\NormalTok{ landsat8 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(country)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)}\OperatorTok{;}  
  
\CommentTok{// Apply scaling factors.  }
\KeywordTok{function} \FunctionTok{applyScaleFactors}\NormalTok{(image) \{   }\KeywordTok{var}\NormalTok{ opticalBands }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SR\_B.\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\FloatTok{0.0000275}\NormalTok{)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(}\OperatorTok{{-}}       \FloatTok{0.2}\NormalTok{)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ thermalBands }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}ST\_B.*\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\FloatTok{0.00341802}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{add}\NormalTok{(}\FloatTok{149.0}\NormalTok{)}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(opticalBands}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(thermalBands}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\}  }
  
\NormalTok{landsat8 }\OperatorTok{=}\NormalTok{ landsat8}\OperatorTok{.}\FunctionTok{map}\NormalTok{(applyScaleFactors)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now, we can create a composite. We will use the median function, which
has the same effect as writing reduce(ee.Reducer.median()) as seen in
Chap. F4.0, to reduce our ImageCollection to a median composite. Add the
resulting composite to the map using visualization parameters.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create composite.  }
\KeywordTok{var}\NormalTok{ composite }\OperatorTok{=}\NormalTok{ landsat8}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(country)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ visParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.2}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(composite}\OperatorTok{,}\NormalTok{ visParams}\OperatorTok{,} \StringTok{\textquotesingle{}L8 Composite\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image12.png}

}

\caption{Fig. F4.3.1 Landsat 8 surface reflectance 2019 median composite
of Colombia}

\end{figure}

The resulting composite (Fig. F4.3.1) has lots of clouds, especially in
the western, mountainous regions of Colombia. In tropical regions, it is
very challenging to generate a high-quality, cloud-free composite
without first filtering images for cloud cover, even if our collection
is constrained to only include images acquired during the dry season.
Therefore, let's filter our collection by the CLOUD\_COVER parameter to
avoid cloudy images. We will start with images that have less than 50\%
cloud cover.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Filter by the CLOUD\_COVER property.  }
\KeywordTok{var}\NormalTok{ landsat8FiltClouds }\OperatorTok{=}\NormalTok{ landsat8  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(country)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{lessThan}\NormalTok{(}\StringTok{\textquotesingle{}CLOUD\_COVER\textquotesingle{}}\OperatorTok{,} \DecValTok{50}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Create a composite from the filtered imagery.  }
\KeywordTok{var}\NormalTok{ compositeFiltClouds }\OperatorTok{=}\NormalTok{ landsat8FiltClouds}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(country)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(compositeFiltClouds}\OperatorTok{,}\NormalTok{ visParams}\OperatorTok{,}   \StringTok{\textquotesingle{}L8 Composite cloud filter\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Print size of collections, for comparison.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Size landsat8 collection\textquotesingle{}}\OperatorTok{,}\NormalTok{ landsat8}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Size landsat8FiltClouds collection\textquotesingle{}}\OperatorTok{,}\NormalTok{ landsat8FiltClouds}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image50.png}

}

\caption{Fig. F4.3.2 Landsat 8 surface reflectance 2019 median composite
of Colombia filtered by cloud cover less than 50\%}

\end{figure}

This new composite (Fig. F4.3.2) looks slightly better than the previous
one, but still very cloudy. Remember to turn off the first layer or
adjust the transparency to visualize only this new composite. The code
prints the size of these collections, using the size function) to see
how many images were left out after we applied the cloud cover
threshold. (There are 1201 images in the landsat8 collection, compared
to 493 in the landsat8FiltClouds collection---a lot of scenes with cloud
cover greater than or equal to 50\%.)

Try adjusting the CLOUD\_COVER threshold in the landsat8FiltClouds
variable to different percentages and checking the results. For example,
with 20\% set as the threshold (Fig. F4.3.3), you can see that many
parts of the country have image gaps. (Remember to turn off the first
layer or adjust its transparency; you can also set the shown parameter
in the Map.addLayer function to false so the layer does not
automatically load). So there is a trade-off between a stricter cloud
cover threshold and data availability. Additionally, even with a cloud
filter, some tiles still present a large area cover of clouds.

\begin{figure}

{\centering \includegraphics{./F4/image42.png}

}

\caption{Fig. F4.3.3 Landsat 8 surface reflectance 2019 median composite
of Colombia filtered by cloud cover less than 20\%}

\end{figure}

This is due to persistent cloud cover in some regions of Colombia.
However, a cloud mask can be applied to improve the results. The Landsat
8 Collection 2 contains a quality assessment (QA) band called QA\_PIXEL
that provides useful information on certain conditions within the data,
and allows users to apply per-pixel filters. Each pixel in the QA band
contains unsigned integers that represent bit-packed combinations of
surface, atmospheric, and sensor conditions.

We will also make use of the QA\_RADSAT band, which indicates which
bands are radiometrically saturated. A pixel value of 1 means saturated,
so we will be masking these pixels.

As described in Chap. F4.0, we will create a function to apply a cloud
mask to an image, and then map this function over our collection. The
mask is applied by using the updateMask function. This function
``eliminates'' undesired pixels from the analysis, i.e., makes them
transparent, by taking the mask as the input. You will see that this
cloud mask function (or similar versions) is used in other chapters of
the book. Note: Remember to set the cloud cover threshold back to 50 in
the landsat8FiltClouds variable.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define the cloud mask function.  }
\KeywordTok{function} \FunctionTok{maskSrClouds}\NormalTok{(image) \{   }\CommentTok{// Bit 0 {-} Fill   // Bit 1 {-} Dilated Cloud   // Bit 2 {-} Cirrus   // Bit 3 {-} Cloud   // Bit 4 {-} Cloud Shadow   var qaMask = image.select(\textquotesingle{}QA\_PIXEL\textquotesingle{}).bitwiseAnd(parseInt(\textquotesingle{}11111\textquotesingle{},       2)).eq(0);   var saturationMask = image.select(\textquotesingle{}QA\_RADSAT\textquotesingle{}).eq(0);   return image.updateMask(qaMask)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(saturationMask)}\OperatorTok{;}  
\NormalTok{\}  }
  
\CommentTok{// Apply the cloud mask to the collection.  }
\KeywordTok{var}\NormalTok{ landsat8FiltMasked }\OperatorTok{=}\NormalTok{ landsat8FiltClouds}\OperatorTok{.}\FunctionTok{map}\NormalTok{(maskSrClouds)}\OperatorTok{;}  
  
\CommentTok{// Create a composite.  }
\KeywordTok{var}\NormalTok{ landsat8compositeMasked }\OperatorTok{=}\NormalTok{ landsat8FiltMasked}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(country)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsat8compositeMasked}\OperatorTok{,}\NormalTok{ visParams}\OperatorTok{,} \StringTok{\textquotesingle{}L8 composite masked\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image39.png}

}

\caption{Fig. F4.3.4 Landsat 8 surface reflectance 2019 median composite
of Colombia filtered by cloud cover less than 50\% and with cloud mask
applied}

\end{figure}

Because we are dealing with bits, in the maskSrClouds function we
utilized the bitwiseAnd and parseInt functions. These are functions that
serve the purpose of unpacking the bit information. A bitwise AND is a
binary operation that takes two equal-length binary representations and
performs the logical AND operation on each pair of corresponding bits.
Thus, if both bits in the compared positions have the value 1, the bit
in the resulting binary representation is 1 (1 × 1 = 1); otherwise, the
result is 0 (1 × 0 = 0 and 0 × 0 = 0). The parseInt function parses a
string argument (in our case, five-character string `11111') and returns
an integer of the specified numbering system, base 2.

The resulting composite (Fig. F4.3.4) shows masked clouds, and is more
spatially exhaustive in coverage compared to previous composites (don't
forget to uncheck the previous layers). This is because, when
compositing all the images into one, we are not taking cloudy pixels
into account anymore; therefore, the resulting pixel is not cloud
covered but an actual representation of the landscape. However, data
gaps are still an issue due to cloud cover. If you do not specifically
need an annual composite, a first approach is to create a two-year
composite to try to mitigate the missing data issue, or to have a series
of rules that allows for selecting pixels for that particular year (as
in Sect. 3 below). Change the startDate variable to 2018-01-01 to
include all images from 2018 and 2019 in the collection. How does the
cloud-masked composite (Fig. F4.3.5) compare to the 2019 one?

\begin{figure}

{\centering \includegraphics{./F4/image85.png}

}

\caption{Fig. F4.3.5 One-year, startDate variable set to 2019-01-01,
(left) and two-year, startDate variable set to 2018-01-01, (right)
median composites with 50\% cloud cover threshold and cloud mask
applied}

\end{figure}

The resulting image has substantially fewer data gaps (you can zoom in
to better see them). Again, if the time period is not a constraint for
the creation of your composite, you can incorporate more images from a
third year, and so on.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F43a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{incorporating-data-from-other-satellites}{%
\subsection{Incorporating Data from Other
Satellites}\label{incorporating-data-from-other-satellites}}

Another option to reduce the presence of data gaps in cloudy situations
is to bring in imagery from other sensors acquired during the time
period of interest. The Landsat collection spans multiple missions,
which have continuously acquired uninterrupted data since 1972 at
different acquisition dates. Next, we will try incorporating Landsat 7
Level 2, Collection 2, Tier 1 images from 2019 to fill the gaps in the
2019 Landsat 8 composite.

To generate a Landsat 7 composite, we apply similar steps to the ones we
did for Landsat 8, so keep adding code to the same script from Sect. 1.
First, define your Landsat 7 collection variable and the scaling
function. Then, filter the collection, apply the cloud mask (since we
know Colombia has persistent cloud cover), and apply the scaling
function. Note that we will use the same cloud mask function defined
above, since the bits information for Landsat 7 is the same as for
Landsat 8. Finally, create the median composite. After pasting in the
code below but before executing it, change the startDate variable back
to 2019-01-01 in order to create a one-year composite of 2019.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// {-}{-}{-}{-}{-}{-}{-}{-}{-}{-} Section 2 {-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}{-}  }
  
\CommentTok{// Define Landsat 7 Level 2, Collection 2, Tier 1 collection.  }
\KeywordTok{var}\NormalTok{ landsat7 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LE07/C02/T1\_L2\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Scaling factors for L7.  }
\KeywordTok{function} \FunctionTok{applyScaleFactorsL7}\NormalTok{(image) \{   }\KeywordTok{var}\NormalTok{ opticalBands }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SR\_B.\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\FloatTok{0.0000275}\NormalTok{)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(}\OperatorTok{{-}}       \FloatTok{0.2}\NormalTok{)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ thermalBand }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}ST\_B6\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\FloatTok{0.00341802}\NormalTok{)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(       }\FloatTok{149.0}\NormalTok{)}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(opticalBands}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(thermalBand}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\}  }
  
\CommentTok{// Filter collection, apply cloud mask, and scaling factors.  }
\KeywordTok{var}\NormalTok{ landsat7FiltMasked }\OperatorTok{=}\NormalTok{ landsat7  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(country)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{lessThan}\NormalTok{(}\StringTok{\textquotesingle{}CLOUD\_COVER\textquotesingle{}}\OperatorTok{,} \DecValTok{50}\NormalTok{))  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(maskSrClouds)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(applyScaleFactorsL7)}\OperatorTok{;}  
  
\CommentTok{// Create composite.  }
\KeywordTok{var}\NormalTok{ landsat7compositeMasked }\OperatorTok{=}\NormalTok{ landsat7FiltMasked  }
   \OperatorTok{.}\FunctionTok{median}\NormalTok{()  }
   \OperatorTok{.}\FunctionTok{clip}\NormalTok{(country)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsat7compositeMasked}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.2}\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}L7 composite masked\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image73.png}

}

\caption{Fig. F4.3.6 One-year Landsat 7 median composite with 50\% cloud
cover threshold and cloud mask applied}

\end{figure}

Note that we used bands: {[}`SR\_B3', `SR\_B2', `SR\_B1'{]} to visualize
the composite because Landsat 7 has different band designations. The
sensors aboard each of the Landsat satellites were designed to acquire
data in different ranges of frequencies along the electromagnetic
spectrum. Whereas for Landsat 8, the red, green, and blue bands are B4,
B3, and B2, respectively, for Landsat 7, these same bands are B3, B2,
and B1, respectively.

You should see an image with systematic gaps like the one shown in Fig.
F4.3.6 (remember to turn off the other layers, and zoom in to better see
the data gaps). Landsat 7 was launched in 1999, but since 2003, the
sensor has acquired and delivered data with data gaps caused by a scan
line corrector (SLC) failure. Without an operating SLC, the sensor's
line of sight traces a zig-zag pattern along the satellite ground track,
and, as a result, the imaged area is duplicated and some areas are
missed. When the Level 1 data are processed, the duplicated areas are
removed, leaving data gaps (Fig. F4.3.7). For more information about
Landsat 7 and SLC error, please refer to the USGS Landsat 7 page.
However, even with the SLC error, we can still use the Landsat 7 data in
our composite. Now, let's combine the Landsat 7 and 8 collections.

\begin{figure}

{\centering \includegraphics{./F4/image54.png}

}

\caption{Fig. F4.3.7 Landsat 7's SLC-off condition. Source: USGS}

\end{figure}

Since Landsat 7 and 8 have different band designations, first we create
a function to rename the bands from Landsat 7 to match the names used
for Landsat 8 and map that function over our Landsat 7 collection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Since Landsat 7 and 8 have different band designations,  }
\CommentTok{// let\textquotesingle{}s create a function to rename L7 bands to match to L8.  }
\KeywordTok{function} \FunctionTok{rename}\NormalTok{(image) \{   }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(  }
\NormalTok{       [}\StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
\NormalTok{       [}\StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\NormalTok{\}  }
  
\CommentTok{// Apply the rename function.  }
\KeywordTok{var}\NormalTok{ landsat7FiltMaskedRenamed }\OperatorTok{=}\NormalTok{ landsat7FiltMasked}\OperatorTok{.}\FunctionTok{map}\NormalTok{(rename)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If you print the first images of both the landsat7FiltMasked and
landsat7FiltMaskedRenamed collections (Fig. F4.3.8), you will see that
the bands got renamed, and not all bands got copied over
(SR\_ATMOS\_OPACITY, SR\_CLOUD\_QA, SR\_B6, etc.). To copy these
additional bands, simply add them to the rename function. You will need
to rename SR\_B6 so it does not have the same name as the new band 5.

\begin{figure}

{\centering \includegraphics{./F4/image15.png}

}

\caption{Fig. F4.3.8 First images of landsat7FiltMasked and
landsat7FiltMaskedRenamed, respectively}

\end{figure}

Now we merge the two collections using the merge function for
ImageCollection and mapping over a function to cast the Landsat 7 input
values to a 32-bit float using the toFloat function for consistency. To
merge collections, the number and names of the bands must be the same in
each collection. We use the select function (Chap. F1.1) to select the
Landsat 8 bands to be the same as Landsat 7's. When creating the new
Landsat 7 and 8 composite, if we did not select these 6 bands, we would
get an error message for trying to composite a collection that has 6
bands (Landsat 7) with a collection that has 19 bands (Landsat 8).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Merge Landsat collections.  }
\KeywordTok{var}\NormalTok{ landsat78 }\OperatorTok{=}\NormalTok{ landsat7FiltMaskedRenamed  }
   \OperatorTok{.}\FunctionTok{merge}\NormalTok{(landsat8FiltMasked}\OperatorTok{.}\FunctionTok{select}\NormalTok{(  }
\NormalTok{       [}\StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\NormalTok{]))  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(img) \{       }\ControlFlowTok{return}\NormalTok{ img}\OperatorTok{.}\FunctionTok{toFloat}\NormalTok{()}\OperatorTok{;}  
\NormalTok{   \})}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Merged collections\textquotesingle{}}\OperatorTok{,}\NormalTok{ landsat78)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now we have a collection with about 1000 images. Next, we will take the
median of the values across the ImageCollection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create Landsat 7 and 8 image composite and add to the Map.  }
\KeywordTok{var}\NormalTok{ landsat78composite }\OperatorTok{=}\NormalTok{ landsat78}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(country)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landsat78composite}\OperatorTok{,}\NormalTok{ visParams}\OperatorTok{,} \StringTok{\textquotesingle{}L7 and L8 composite\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Comparing the composite generated considering both Landsat 7 and 8 to
the Landsat 8-only composite, it is evident that there is a reduction in
the amount of data gaps in the final result (Fig. F4.3.9). The resulting
Landsat 7 and 8 image composite still has data gaps due to the presence
of clouds and Landsat 7's SLC-off data. You can try setting the center
of the map to the point with latitude 3.6023 and longitude −75.0741 to
see the inset example of Fig. F4.3.9.

\begin{figure}

{\centering \includegraphics{./F4/image30.png}

}

\caption{Fig. F4.3.9 Landsat 8-only composite (left) and Landsat 7 and 8
composite (right) for 2019. Inset centered at latitude 3.6023, longitude
−75.0741.}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F43b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{best-available-pixel-compositing-earth-engine-application}{%
\subsection{Best-Available-Pixel Compositing Earth Engine
Application}\label{best-available-pixel-compositing-earth-engine-application}}

This section presents an Earth Engine application that enables the
generation of annual best-available-pixel (BAP) image composites by
globally combining multiple Landsat sensors and images: GEE-BAP. Annual
BAP image composites are generated by choosing optimal observations for
each pixel from all available Landsat 5 TM, Landsat 7 ETM+, and Landsat
8 OLI imagery within a given year and within a given day range from a
specified acquisition day of year, in addition to other constraints
defined by the user. The data accessible via Earth Engine are from the
USGS free and open archive of Landsat data. The Landsat images used are
atmospherically corrected to surface reflectance values. Following White
et al.~(2014), a series of scoring functions ranks each pixel
observation for (1) acquisition day of year, (2) cloud cover in the
scene, (3) distance to clouds and cloud shadows, (4) presence of haze,
and (5) acquisition sensor. Further information on the BAP image
compositing approach can be found in Griffiths et al.~(2013), and
detailed information on tuning parameters can be found in White et
al.~(2014).

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F43c. The book's repository contains information about
accessing the GEE-BAP interface and its related functions.

\end{tcolorbox}

\includegraphics{./F4/image70.png}\includegraphics{./F4/image81.png}\includegraphics{./F4/image83.png}

Once you have loaded the GEE-BAP interface (Fig. F4.3.10) using the
instructions in the Code Checkpoint, you will notice that it is divided
into three sections: (1) Input/Output options, (2) Pixel scoring
options, and (3) Advanced parameters. Users indicate the study area, the
time period for generating annual BAP composites (i.e., start and end
years), and the path to store the results in the Input/Output options.
Users have three options to define the study area. The Draw study area
option uses the Draw a shape and Draw a rectangle tools to define the
area of interest. The Upload image template option utilizes an image
template uploaded by the user in TIFF format. This option is well suited
to generating BAP composites that match the projection, pixel size, and
extent to existing raster datasets. The Work globally option generates
BAP composites for the entire globe; note that when this option is
selected, complete data download is not available due to the Earth's
size. With Start year and End year, users can indicate the beginning and
end of the annual time series of BAP image composites to be generated.
Multiple image composites are then generated---one composite for each
year---resulting in a time series of annual composites. For each year,
composites are uniquely generated utilizing images acquired on the days
within the specified Date range. Produced BAP composites can be saved in
the indicated (Path) Google Drive folder using the Tasks tab. Results
are generated in a tiled, TIFF format, accompanied by a CSV file that
indicates the parameters used to construct the composite.

As noted, GEE-BAP implements five pixel scoring functions: (1) target
acquisition day of year and day range, (2) maximum cloud coverage per
scene, (3) distance to clouds and cloud shadows, (4) atmospheric
opacity, and (5) a penalty for images acquired under the Landsat 7 ETM+
SLC-off malfunction. By defining the Acquisition day of year and Day
range, those candidate pixels acquired closer to a defined acquisition
day of year are ranked higher. Note that pixels acquired outside the day
range window are excluded from subsequent composite development. For
example, if the target day of year is defined as ``08-01'' and the day
range as ``31,'' only those pixels acquired between July 1 and August 31
are considered, and the ones acquired closer to August 1 will receive a
higher score.

The scoring function Max cloud cover in scene indicates the maximum
percentage of cloud cover in an image that will be accepted by the user
in the BAP image compositing process. Defining a value of 70\% implies
that only those scenes with less than or equal to 70\% cloud cover will
be considered as a candidate for compositing.

The Distance to clouds and cloud shadows scoring function enables the
user to exclude those pixels identified to contain clouds and shadows by
the QA mask from the generated BAP, as well as decreasing a pixel's
score if the pixel is within a specified proximity of a cloud or cloud
shadow.

The Atmospheric opacity scoring function ranks pixels based on their
atmospheric opacity values, which are indicative of hazy imagery. Pixels
with opacity values that exceed a defined haze expectation (Max opacity)
are excluded. Pixels with opacity values lower than a defined value (Min
opacity) get the maximum score. Pixels with values in between these
limits are scored following the functions defined by Griffiths et
al.~(2013). This scoring function is available only for Landsat 5 TM and
Landsat 7 ETM+ imagery, which provides the opacity attribute in the
image metadata file.

Finally, there is a Landsat 7 ETM+ SLC-off penalty scoring function that
de-emphasizes images acquired following the ETM+ SLC-off malfunction in
2003. The aim of this scoring element is to ensure that TM or OLI data,
which do not have stripes, take precedence over ETM+ when using dates
after the SLC failure. This allows users to avoid the inclusion of
multiple discontinuous small portions of images being used to produce
the BAP image composites, thus reducing the spatial variability of the
spectral data. The penalty applied to SLC-off imagery is defined
directly proportional to the overall score. A large score reduces the
chance that SLC-off imagery will be used in the composite. A value of 1
prevents SLC-off imagery from being used.

By default, the GEE-BAP application produces image composites using all
the visible bands. The Spectral index option enables the user to produce
selected spectral indices from the resulting BAP image composites.
Available spectral indices include: Normalized Difference Vegetation
Index (NDVI, Fig. F4.3.11), Enhanced Vegetation Index (EVI), and
Normalized Burn Ratio (NBR), as well as several indices derived from the
Tasseled Cap transformation: Wetness (TCW), Greenness (TCG), Brightness
(TCB), and Angle (TCA). Composited indices are able to be downloaded as
well as viewed on the map.

\begin{figure}

{\centering \includegraphics{./F4/image60.png}

}

\caption{Fig. F4.3.11 Example of a global BAP image composite showing
NDVI values generated using the GEE-BAP user interface}

\end{figure}

GEE-BAP functions can be accessed programmatically, including pixel
scoring parameters, as well as BAP image compositing (BAP), de-spiking
(despikeCollection), data-gap infilling (infill), and displaying
(ShowCollection) functions. The following code sets the scoring
parameter values, then generates and displays the compositing results
(Fig. F4.3.12) for a BAP composite that is de-spiked, with data gaps
infilled using temporal interpolation. Copy and paste the code below
into a new script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define required parameters.  }
\KeywordTok{var}\NormalTok{ targetDay }\OperatorTok{=} \StringTok{\textquotesingle{}06{-}01\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ daysRange }\OperatorTok{=} \DecValTok{75}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ cloudsTh }\OperatorTok{=} \DecValTok{70}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ SLCoffPenalty }\OperatorTok{=} \FloatTok{0.7}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ opacityScoreMin }\OperatorTok{=} \FloatTok{0.2}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ opacityScoreMax }\OperatorTok{=} \FloatTok{0.3}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ cloudDistMax }\OperatorTok{=} \DecValTok{1500}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ despikeTh }\OperatorTok{=} \FloatTok{0.65}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ despikeNbands }\OperatorTok{=} \DecValTok{3}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ startYear }\OperatorTok{=} \DecValTok{2015}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endYear }\OperatorTok{=} \DecValTok{2017}\OperatorTok{;}  
  
\CommentTok{// Define study area.  }
\KeywordTok{var}\NormalTok{ worldCountries }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}USDOS/LSIB\_SIMPLE/2017\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ colombia }\OperatorTok{=}\NormalTok{ worldCountries}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{\textquotesingle{}country\_na\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}Colombia\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Load the bap library.  }
\KeywordTok{var}\NormalTok{ library }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/sfrancini/bap:library\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Calculate BAP.  }
\KeywordTok{var}\NormalTok{ BAPCS }\OperatorTok{=}\NormalTok{ library}\OperatorTok{.}\FunctionTok{BAP}\NormalTok{(}\KeywordTok{null}\OperatorTok{,}\NormalTok{ targetDay}\OperatorTok{,}\NormalTok{ daysRange}\OperatorTok{,}\NormalTok{ cloudsTh}\OperatorTok{,}  
\NormalTok{   SLCoffPenalty}\OperatorTok{,}\NormalTok{ opacityScoreMin}\OperatorTok{,}\NormalTok{ opacityScoreMax}\OperatorTok{,}\NormalTok{ cloudDistMax)}\OperatorTok{;}  
  
\CommentTok{// Despike the collection.  }
\NormalTok{BAPCS }\OperatorTok{=}\NormalTok{ library}\OperatorTok{.}\FunctionTok{despikeCollection}\NormalTok{(despikeTh}\OperatorTok{,}\NormalTok{ despikeNbands}\OperatorTok{,}\NormalTok{ BAPCS}\OperatorTok{,}   \DecValTok{1984}\OperatorTok{,} \DecValTok{2021}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Infill datagaps.  }
\NormalTok{BAPCS }\OperatorTok{=}\NormalTok{ library}\OperatorTok{.}\FunctionTok{infill}\NormalTok{(BAPCS}\OperatorTok{,} \DecValTok{1984}\OperatorTok{,} \DecValTok{2021}\OperatorTok{,} \KeywordTok{false}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Visualize the image.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(colombia}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}  
\NormalTok{library}\OperatorTok{.}\FunctionTok{ShowCollection}\NormalTok{(BAPCS}\OperatorTok{,}\NormalTok{ startYear}\OperatorTok{,}\NormalTok{ endYear}\OperatorTok{,}\NormalTok{ colombia}\OperatorTok{,} \KeywordTok{false}\OperatorTok{,}   \KeywordTok{null}\NormalTok{)}\OperatorTok{;}  
\NormalTok{library}\OperatorTok{.}\FunctionTok{AddSLider}\NormalTok{(startYear}\OperatorTok{,}\NormalTok{ endYear)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image11.png}

}

\caption{Fig. F4.3.12 Outcome of the compositing code}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F43d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-10}{%
\subsection*{Conclusion}\label{conclusion-10}}
\addcontentsline{toc}{subsection}{Conclusion}

We cannot monitor what we cannot see. Image compositing algorithms
provide robust and transparent tools to address issues with clouds,
cloud shadows, haze, and smoke in remotely sensed images derived from
optical satellite data, and expand data availability for remote sensing
applications. The tools and approaches described here should provide you
with some useful strategies to aid in mitigating the presence of cloud
cover in your data. Note that the quality of image outcomes is a
function of the quality of cloud masking routines applied to the source
data to generate the various flags that are used in the scoring
functions described herein. Different compositing parameters can be used
to represent a given location as a function of conditions that are
present at a given point in time and the information needs of the end
user. Tuning or optimization of compositing parameters is possible (and
recommended) to ensure best capture of the physical conditions of
interest.

\hypertarget{references-6}{%
\subsection*{References}\label{references-6}}
\addcontentsline{toc}{subsection}{References}

Braaten JD, Cohen WB, Yang Z (2015) Automated cloud and cloud shadow
identification in Landsat MSS imagery for temperate ecosystems. Remote
Sens Environ 169:128--138. https://doi.org/10.1016/j.rse.2015.08.006

Cao R, Chen Y, Chen J, et al (2020) Thick cloud removal in Landsat
images based on autoregression of Landsat time-series data. Remote Sens
Environ 249:112001. https://doi.org/10.1016/j.rse.2020.112001

Eberhardt IDR, Schultz B, Rizzi R, et al (2016) Cloud cover assessment
for operational crop monitoring systems in tropical areas. Remote Sens
8:219. https://doi.org/10.3390/rs8030219

Griffiths P, van der Linden S, Kuemmerle T, Hostert P (2013) A
pixel-based Landsat compositing algorithm for large area land cover
mapping. IEEE J Sel Top Appl Earth Obs Remote Sens 6:2088--2101.
https://doi.org/10.1109/JSTARS.2012.2228167

Hermosilla T, Wulder MA, White JC, Coops NC (2019) Prevalence of
multiple forest disturbances and impact on vegetation regrowth from
interannual Landsat time series (1985--2015). Remote Sens Environ
233:111403.
\hspace{0pt}\hspace{0pt}\href{https://www.google.com/url?q=https://doi.org/10.1016/j.rse.2019.111403\&sa=D\&source=editors\&ust=1671458868185432\&usg=AOvVaw0FmmtQ4YjpNcI-HVcDozZ-}{https://doi.org/10.1016/j.rse.2019.111403}

Hermosilla T, Wulder MA, White JC, et al (2015) An integrated Landsat
time series protocol for change detection and generation of annual
gap-free surface reflectance composites. Remote Sens Environ
158:220--234. https://doi.org/10.1016/j.rse.2014.11.005

Hermosilla T, Wulder MA, White JC, et al (2016) Mass data processing of
time series Landsat imagery: Pixels to data products for forest
monitoring. Int J Digit Earth 9:1035--1054.
https://doi.org/10.1080/17538947.2016.1187673

Huang C, Thomas N, Goward SN, et al (2010) Automated masking of cloud
and cloud shadow for forest change analysis using Landsat images. Int J
Remote Sens 31:5449--5464. https://doi.org/10.1080/01431160903369642

Irish RR, Barker JL, Goward SN, Arvidson T (2006) Characterization of
the Landsat-7 ETM+ automated cloud-cover assessment (ACCA) algorithm.
Photogramm Eng Remote Sensing 72:1179--1188.
\href{https://www.google.com/url?q=https://doi.org/10.14358/PERS.72.10.1179\&sa=D\&source=editors\&ust=1671458868186784\&usg=AOvVaw0EDp98fBocaD-wesBTd-lm}{https://doi.org/10.14358/PERS.72.10.1179}

Kennedy RE, Yang Z, Cohen WB (2010) Detecting trends in forest
disturbance and recovery using yearly Landsat time series: 1. LandTrendr
- Temporal segmentation algorithms. Remote Sens Environ 114:2897--2910.
https://doi.org/10.1016/j.rse.2010.07.008

Loveland TR, Dwyer JL (2012) Landsat: Building a strong future. Remote
Sens Environ 122:22--29. https://doi.org/10.1016/j.rse.2011.09.022

Marshall GJ, Rees WG, Dowdeswell JA (1993) Limitations imposed by cloud
cover on multitemporal visible band satellite data sets from polar
regions. Ann Glaciol 17:113--120.
https://doi.org/10.3189/S0260305500012696

Marshall GJ, Dowdeswell JA, Rees WG (1994) The spatial and temporal
effect of cloud cover on the acquisition of high quality landsat imagery
in the European Arctic sector. Remote Sens Environ 50:149--160.
https://doi.org/10.1016/0034-4257(94)90041-8

Martins VS, Novo EMLM, Lyapustin A, et al (2018) Seasonal and
interannual assessment of cloud cover and atmospheric constituents
across the Amazon (2000--2015): Insights for remote sensing and climate
analysis. ISPRS J Photogramm Remote Sens 145:309--327.
https://doi.org/10.1016/j.isprsjprs.2018.05.013

Roberts D, Mueller N, McIntyre A (2017) High-dimensional pixel
composites from Earth observation time series. IEEE Trans Geosci Remote
Sens 55:6254--6264. https://doi.org/10.1109/TGRS.2017.2723896

Roy DP, Ju J, Kline K, et al (2010) Web-enabled Landsat data (WELD):
Landsat ETM+ composited mosaics of the conterminous United States.
Remote Sens Environ 114:35--49.
https://doi.org/10.1016/j.rse.2009.08.011

Sano EE, Ferreira LG, Asner GP, Steinke ET (2007) Spatial and temporal
probabilities of obtaining cloud-free Landsat images over the Brazilian
tropical savanna. Int J Remote Sens 28:2739--2752.
https://doi.org/10.1080/01431160600981517

White JC, Wulder MA, Hobart GW, et al (2014) Pixel-based image
compositing for large-area dense time series applications and science.
Can J Remote Sens 40:192--212.
https://doi.org/10.1080/07038992.2014.945827

Zhu X, Helmer EH (2018) An automatic method for screening clouds and
cloud shadows in optical satellite image time series in cloudy regions.
Remote Sens Environ 214:135--153.
https://doi.org/10.1016/j.rse.2018.05.024

Zhu Z, Wang S, Woodcock CE (2015) Improvement and expansion of the Fmask
algorithm: Cloud, cloud shadow, and snow detection for Landsats 4--7, 8,
and Sentinel 2 images. Remote Sens Environ 159:269--277.
https://doi.org/10.1016/j.rse.2014.12.014

Zhu Z, Woodcock CE (2014) Automated cloud, cloud shadow, and snow
detection in multitemporal Landsat data: An algorithm designed
specifically for monitoring land cover change. Remote Sens Environ
152:217--234. https://doi.org/10.1016/j.rse.2014.06.012

Zhu Z, Woodcock CE (2012) Object-based cloud and cloud shadow detection
in Landsat imagery. Remote Sens Environ 118:83--94.
https://doi.org/10.1016/j.rse.2011.10.028

\hypertarget{change-detection}{%
\section{Change Detection}\label{change-detection}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-9}{%
\subsubsection*{Author}\label{author-9}}

Karis Tenneson, John Dilger, Crystal Wespestad, Brian Zutta, Andréa P
Nicolau, Karen Dyson, Paula Paz

\hypertarget{overview-11}{%
\subsubsection*{Overview}\label{overview-11}}

This chapter introduces change detection mapping. It will teach you how
to make a two-date land cover change map using image differencing and
threshold-based classification. You will use what you have learned so
far in this book to produce a map highlighting changes in the land cover
between two time steps. You will first explore differences between the
two images extracted from these time steps by creating a difference
layer. You will then learn how to directly classify change based on the
information in both of your images.

\hypertarget{learning-outcomes-11}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-11}}

\begin{itemize}
\tightlist
\item
  Creating and exploring how to read a false-color cloud-free Landsat
  composite image
\item
  Calculating the Normalized Burn Ratio (NBR) index.\\
\item
  Creating a two-image difference to help locate areas of change.
\item
  Producing a change map and classifying changes using thresholding.
\end{itemize}

\hypertarget{assumes-you-know-how-to-11}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-11}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-9}{%
\subsection*{Introduction}\label{introduction-9}}

Change detection is the process of assessing how landscape conditions
are changing by looking at differences in images acquired at different
times. This can be used to quantify changes in forest cover---such as
those following a volcanic eruption, logging activity, or wildfire---or
when crops are harvested (Fig. F4.4.1). For example, using time-series
change detection methods, Hansen et al.~(2013) quantified annual changes
in forest loss and regrowth. Change detection mapping is important for
observing, monitoring, and quantifying changes in landscapes over time.
Key questions that can be answered using these techniques include
identifying whether a change has occurred, measuring the area or the
spatial extent of the region undergoing change, characterizing the
nature of the change, and measuring the pattern (configuration or
composition) of the change (MacLeod and Congalton 1998).

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\tightlist
\item
\end{enumerate}

\includegraphics{./F4/image80.png}

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\setcounter{enumi}{1}
\tightlist
\item
\end{enumerate}

\includegraphics{./F4/image21.png}

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\setcounter{enumi}{2}
\tightlist
\item
\end{enumerate}

\includegraphics{./F4/image75.png}

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\setcounter{enumi}{3}
\tightlist
\item
\end{enumerate}

\begin{figure}

{\centering \includegraphics{./F4/image87.png}

}

\caption{Fig. F4.4.1 Before and after images of (a) the eruption of
Mount St.~Helens in Washington State, USA, in 1980 (before, July 10,
1979; after, September 5, 1980); (b) the Camp Fire in California, USA,
in 2018 (before, October 7, 2018; after, March 16, 2019); (c) illegal
gold mining in the Madre de Dios region of Peru (before, March 31, 2001;
after, August 22, 2020); and (d) shoreline changes in Incheon, South
Korea (before, May 29, 1981; after, March 11, 2020)}

\end{figure}

Many change detection techniques use the same basic premise: that most
changes on the landscape result in spectral values that differ between
pre-event and post-event images. The challenge can be to separate the
real changes of interest---those due to activities on the
landscape---from noise in the spectral signal, which can be caused by
seasonal variation and phenology, image misregistration, clouds and
shadows, radiometric inconsistencies, variability in illumination (e.g.,
sun angle, sensor position), and atmospheric effects.

Activities that result in pronounced changes in radiance values for a
sufficiently long time period are easier to detect using remote sensing
change detection techniques than are subtle or short-lived changes in
landscape conditions. Mapping challenges can arise if the change event
is short-lived, as these are difficult to capture using satellite
instruments that only observe a location every several days. Other types
of changes occur so slowly or are so vast that they are not easily
detected until they are observed using satellite images gathered over a
sufficiently long interval of time. Subtle changes that occur slowly on
the landscape may be better suited to more computationally demanding
methods, such as time-series analysis. Kennedy et al.~(2009) provides a
nice overview of the concepts and tradeoffs involved when designing
landscape monitoring approaches. Additional summaries of change
detection methods and recent advances include Singh (1989), Coppin et
al.~(2004), Lu et al.~(2004), and Woodcock et al.~(2020).

For land cover changes that occur abruptly over large areas on the
landscape and are long-lived, a simple two-date image differencing
approach is suitable. Two-date image differencing techniques are
long-established methods for identifying changes that produce easily
interpretable results (Singh 1989). The process typically involves four
steps: (1) image selection and preprocessing; (2) data transformation,
such as calculating the difference between indices of interest (e.g.,
the Normalized Difference Vegetation Index (NDVI)) in the pre-event and
post-event images; (3) classifying the differenced image(s) using
thresholding or supervised classification techniques; and (4)
evaluation.

For the practicum, you will select pre-event and post-event image scenes
and investigate the conditions in these images in a false-color
composite display. Next, you will calculate the NBR index for each scene
and create a difference image using the two NBR maps. Finally, you will
apply a threshold to the difference image to establish categories of
changed versus stable areas (Fig. F4.4.2).

\begin{figure}

{\centering \includegraphics{./F4/image68.png}

}

\caption{Fig. F4.4.2 Change detection workflow for this practicum}

\end{figure}

\hypertarget{preparing-imagery}{%
\subsection{Preparing Imagery}\label{preparing-imagery}}

Before beginning a change detection workflow, image preprocessing is
essential. The goal is to ensure that each pixel records the same type
of measurement at the same location over time. These steps include
multitemporal image registration and radiometric and atmospheric
corrections, which are especially important. A lot of this work has been
automated and already applied to the images that are available in Earth
Engine. Image selection is also important. Selection considerations
include finding images with low cloud cover and representing the same
phenology (e.g., leaf-on or leaf-off).

The code in the block below accesses the USGS Landsat 8 Level 2,
Collection 2, Tier 1 dataset and assigns it to the variable landsat8. To
improve readability when working with the Landsat 8 ImageCollection, the
code selects bands 2--7 and renames them to band names instead of band
numbers.

var landsat8 = ee.ImageCollection(`LANDSAT/LC08/C02/T1\_L2')\\
.select(\\
{[}`SR\_B2', `SR\_B3', `SR\_B4', `SR\_B5', `SR\_B6', `SR\_B7'{]},\\
{[}`blue', `green', `red', `nir', `swir1', `swir2'{]}\\
);

Next, you will split the Landsat 8 ImageCollection into two collections,
one for each time period, and apply some filtering and sorting to get an
image for each of two time periods. In this example, we know there are
few clouds for the months of the analysis; if you're working in a
different area, you may need to apply some cloud masking or mosaicing
techniques (see Chap. F4.3).

The code below does several things. First, it creates a new geometry
variable to filter the geographic bounds of the image collections. Next,
it creates a new variable for the pre-event image by (1) filtering the
collection by the date range of interest (e.g., June 2013), (2)
filtering the collection by the geometry, (3) sorting by cloud cover so
the first image will have the least cloud cover, and (4) getting the
first image from the collection.

Now repeat the previous step, but assign it to a post-event image
variable and change the filter date to a period after the pre-event
image's date range (e.g., June 2020).

var point = ee.Geometry.Point({[}-123.64, 42.96{]});\\
Map.centerObject(point, 11);

var preImage = landsat8\\
.filterBounds(point)\\
.filterDate(`2013-06-01', `2013-06-30')\\
.sort(`CLOUD\_COVER', true)\\
.first(); var postImage = landsat8\\
.filterBounds(point)\\
.filterDate(`2020-06-01', `2020-06-30')\\
.sort(`CLOUD\_COVER', true)\\
.first();

\hypertarget{creating-false-color-composites}{%
\subsection{Creating False-Color
Composites}\label{creating-false-color-composites}}

Before running any sort of change detection analysis, it is useful to
first visualize your input images to get a sense of the landscape,
visually inspect where changes might occur, and identify any problems in
the inputs before moving further. As described in Chap. F1.1,
false-color composites draw bands from multispectral sensors in the red,
green, and blue channels in ways that are designed to illustrate
contrast in imagery. Below, you will produce a false-color composite
using SWIR-2 in the red channel, NIR in the green channel, and Red in
the blue channel (Fig. F4.4.3).

Following the format in the code block below, first create a variable
visParam to hold the display parameters, selecting the SWIR-2, NIR, and
red bands, with values drawn that are between 7750 and 22200. Next, add
the pre-event and post-event images to the map and click Run. Click and
drag the opacity slider on the post-event image layer back and forth to
view the changes between your two images.

var visParam = \{ `bands': {[}`swir2', `nir', `red'{]}, `min': 7750,
`max': 22200\\
\};\\
Map.addLayer(preImage, visParam, `pre');\\
Map.addLayer(postImage, visParam, `post');

\begin{figure}

{\centering \includegraphics{./F4/image31.png}

}

\caption{Fig. F4.4.3 False-color composite using SWIR2, NIR, and red.
Vegetation shows up vividly in the green channel due to vegetation being
highly reflective in the NIR band. Shades of green can be indicative of
vegetation density; water typically shows up as black to dark blue; and
burned or barren areas show up as brown.}

\end{figure}

\hypertarget{calculating-nbr}{%
\subsection{Calculating NBR}\label{calculating-nbr}}

The next step is data transformation, such as calculating NBR. The
advantage of using these techniques is that the data, along with the
noise inherent in the data, have been reduced in order to simplify a
comparison between two images. Image differencing is done by subtracting
the spectral value of the first-date image from that of the second-date
image, pixel by pixel (Fig. F4.4.2). Two-date image differencing can be
used with a single band or with spectral indices, depending on the
application. Identifying the correct band or index to identify change
and finding the correct thresholds to classify it are critical to
producing meaningful results. Working with indices known to highlight
the land cover conditions before and after a change event of interest is
a good starting point. For example, the Normalized Difference Water
Index would be good for mapping water level changes during flooding
events; the NBR is good at detecting soil brightness; and the NDVI can
be used for tracking changes in vegetation (although this index does
saturate quickly). In some cases, using derived band combinations that
have been customized to represent the phenomenon of interest is
suggested, such as using the Normalized Difference Fraction Index to
monitor forest degradation (see Chap. A3.4).

Examine changes to the landscape caused by fires using NBR, which
measures the severity of fires using the equation (NIR − SWIR) / (NIR +
SWIR). These bands were chosen because they respond most strongly to the
specific changes in forests caused by fire. This type of equation, a
difference of variables divided by their sum, is referred to as a
normalized difference equation (see Chap. F2.0). The resulting value
will always fall between −1 and 1. NBR is useful for determining whether
a fire recently occurred and caused damage to the vegetation, but it is
not designed to detect other types of land cover changes especially
well.

First, calculate the NBR for each time period using the built-in
normalized difference function. For Landsat 8, be sure to utilize the
NIR and SWIR2 bands to calculate NBR. Then, rename each image band with
the built-in rename function.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Calculate NBR.  }
\KeywordTok{var}\NormalTok{ nbrPre }\OperatorTok{=}\NormalTok{ preImage}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}nir\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}swir2\textquotesingle{}}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}nbr\_pre\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ nbrPost }\OperatorTok{=}\NormalTok{ postImage}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}nir\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}swir2\textquotesingle{}}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}nbr\_post\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F44a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{single-date-transformation}{%
\subsection{Single Date
Transformation}\label{single-date-transformation}}

Next, we will examine the changes that have occurred, as seen when
comparing two specific dates in time.

Subtract the pre-event image from the post-event image using the
subtract function. Add the two-date change image to the map with the
specialized Fabio Crameri batlow color ramp (Crameri et al.~2020). This
color ramp is an example of a color combination specifically designed to
be readable by colorblind and color-deficient viewers. Being cognizant
of your cartographic choices is an important part of making a good
change map.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// 2{-}date change.  }
\KeywordTok{var}\NormalTok{ diff }\OperatorTok{=}\NormalTok{ nbrPost}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(nbrPre)}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}change\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ palette }\OperatorTok{=}\NormalTok{ [   }\StringTok{\textquotesingle{}011959\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}0E365E\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1D5561\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}3E6C55\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}687B3E\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}9B882E\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}D59448\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}F9A380\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}FDB7BD\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}FACCFA\textquotesingle{}}  
\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ visParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palette}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.2}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.2}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(diff}\OperatorTok{,}\NormalTok{ visParams}\OperatorTok{,} \StringTok{\textquotesingle{}change\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Question 1. Try to interpret the resulting image before reading on. What
patterns of change can you identify? Can you find areas that look like
vegetation loss or gain?

The color ramp has dark blues for the lowest values, greens and oranges
in the midrange, and pink for the highest values. We used nbrPre
subtracted from nbrPost to identify changes in each pixel. Since NBR
values are higher when vegetation is present, areas that are negative in
the change image will represent pixels that were higher in the nbrPre
image than in the nbrPost image. Conversely, positive differences mean
that an area gained vegetation (Fig. F4.4.4).

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\item
\begin{verbatim}
                                       b)                                           c)
\end{verbatim}
\end{enumerate}

\begin{figure}

{\centering \includegraphics{./F4/image4.png}

}

\caption{Fig. F4.4.4 (a) Two-date NBR difference; (b) pre-event image
(June 2013) false-color composite; (c) post-event image (June 2020)
false-color composite. In the change map (a), areas on the lower range
of values (blue) depict areas where vegetation has been negatively
affected, and areas on the higher range of values (pink) depict areas
where there has been vegetation gain; the green/orange areas have
experienced little change. In the pre-event and post-event images (b and
c), the green areas indicate vegetation, while the brown regions are
barren ground.}

\end{figure}

\hypertarget{classifying-change}{%
\subsection{Classifying Change}\label{classifying-change}}

Once the images have been transformed and differenced to highlight areas
undergoing change, the next step is image classification into a thematic
map consisting of stable and change classes. This can be done rather
simply by thresholding the change layer, or by using classification
techniques such as machine learning algorithms. One challenge of working
with simple thresholding of the difference layer is knowing how to
select a suitable threshold to partition changed areas from stable
classes. On the other hand, classification techniques using machine
learning algorithms partition the landscape using examples of reference
data that you provide to train the classifier. This may or may not yield
better results, but does require additional work to collect reference
data and train the classifier. In the end, resources, timing, and the
patterns of the phenomenon you are trying to map will determine which
approach is suitable---or perhaps the activity you are trying to track
requires something more advanced, such as a time-series approach that
uses more than two dates of imagery.

For this chapter, we will classify our image into categories using a
simple, manual thresholding method, meaning we will decide the optimal
values for when a pixel will be considered change or no-change in the
image. Finding the ideal value is a considerable task and will be unique
to each use case and set of inputs (e.g., the threshold values for a
SWIR2 single-band change would be different from the thresholds for
NDVI). For a look at a more advanced method of thresholding, check out
the thresholding methods in Chap. A2.3.

First, you will define two variables for the threshold values for gain
and loss. Next, create a new image with a constant value of 0. This will
be the basis of our classification. Reclassify the new image using the
where function. Classify loss areas as 2 where the difference image is
less than or equal to the loss threshold value. Reclassify gain areas to
1 where the difference image is greater than or equal to the gain
threshold value. Finally, mask the image by itself and add the
classified image to the map (Fig. F4.4.5). Note: It is not necessary to
self-mask the image, and in many cases you might be just as interested
in areas that did not change as you are in areas that did.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Classify change  }
\KeywordTok{var}\NormalTok{ thresholdGain }\OperatorTok{=} \FloatTok{0.10}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ thresholdLoss }\OperatorTok{=} \OperatorTok{{-}}\FloatTok{0.10}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ diffClassified }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{;}  
  
\NormalTok{diffClassified }\OperatorTok{=}\NormalTok{ diffClassified}\OperatorTok{.}\FunctionTok{where}\NormalTok{(diff}\OperatorTok{.}\FunctionTok{lte}\NormalTok{(thresholdLoss)}\OperatorTok{,} \DecValTok{2}\NormalTok{)}\OperatorTok{;}  
\NormalTok{diffClassified }\OperatorTok{=}\NormalTok{ diffClassified}\OperatorTok{.}\FunctionTok{where}\NormalTok{(diff}\OperatorTok{.}\FunctionTok{gte}\NormalTok{(thresholdGain)}\OperatorTok{,} \DecValTok{1}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ changeVis }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:} \StringTok{\textquotesingle{}fcffc8,2659eb,fa1373\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{2}  
\NormalTok{\}}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(diffClassified}\OperatorTok{.}\FunctionTok{selfMask}\NormalTok{()}\OperatorTok{,}  
\NormalTok{   changeVis}\OperatorTok{,}   \StringTok{\textquotesingle{}change classified by threshold\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\tightlist
\item
\end{enumerate}

\includegraphics{./F4/image77.png}\includegraphics{./F4/image77.png}

\begin{enumerate}
\def\labelenumi{\alph{enumi})}
\setcounter{enumi}{1}
\tightlist
\item
\end{enumerate}

\includegraphics{./F4/image17.png}\includegraphics{./F4/image17.png}

Chapters F4.5 through F4.9 present more-advanced change detection
algorithms that go beyond differencing and thresholding between two
images, instead allowing you to analyze changes indicated across several
images as a time series.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F44b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-11}{%
\subsection*{Conclusion}\label{conclusion-11}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to make a change detection map using
two-image differencing. The importance of visualizing changes in this
way instead of using a post-classification comparison, where two
classified maps are compared instead of two satellite images, is that it
avoids multiplicative errors from the classifications and is better at
observing more subtle changes in the landscape. You also learned that
how you visualize your images and change maps---such as what band
combinations and color ramps you select, and what threshold values you
use for a classification map---has an impact on how easily and what
types of changes can be seen.

\hypertarget{references-7}{%
\subsection*{References}\label{references-7}}
\addcontentsline{toc}{subsection}{References}

Cohen WB, Healey SP, Yang Z, et al (2017) How similar are forest
disturbance maps derived from different Landsat time series algorithms?
Forests 8:98. https://doi.org/10.3390/f8040098

Coppin P, Jonckheere I, Nackaerts K, et al (2004) Digital change
detection methods in ecosystem monitoring: A review. Int J Remote Sens
25:1565--1596. https://doi.org/10.1080/0143116031000101675

Crameri F, Shephard GE, Heron PJ (2020) The misuse of colour in science
communication. Nat Commun 11:1--10.
https://doi.org/10.1038/s41467-020-19160-7

Fung T (1990) An assessment of TM imagery for land-cover change
detection. IEEE Trans Geosci Remote Sens 28:681--684.
https://doi.org/10.1109/TGRS.1990.572980

Hansen MC, Potapov PV, Moore R, et al (2013) High-resolution global maps
of 21st-century forest cover change. Science 342:850--853.
https://doi.org/10.1126/science.1244693

Kennedy RE, Townsend PA, Gross JE, et al (2009) Remote sensing change
detection tools for natural resource managers: Understanding concepts
and tradeoffs in the design of landscape monitoring projects. Remote
Sens Environ 113:1382--1396. https://doi.org/10.1016/j.rse.2008.07.018

Lu D, Mausel P, Brondízio E, Moran E (2004) Change detection techniques.
Int J Remote Sens 25:2365--2401.
https://doi.org/10.1080/0143116031000139863

Macleod RD, Congalton RG (1998) A quantitative comparison of
change-detection algorithms for monitoring eelgrass from remotely sensed
data. Photogramm Eng Remote Sensing 64:207--216

Singh A (1989) Digital change detection techniques using remotely-sensed
data. Int J Remote Sens 10:989--1003.
https://doi.org/10.1080/01431168908903939

Stehman SV, Czaplewski RL (1998) Design and analysis for thematic map
accuracy assessment: Fundamental principles. Remote Sens Environ
64:331--344. https://doi.org/10.1016/S0034-4257(98)00010-8

Woodcock CE, Loveland TR, Herold M, Bauer ME (2020) Transitioning from
change detection to monitoring with remote sensing: A paradigm shift.
Remote Sens Environ 238:111558.
https://doi.org/10.1016/j.rse.2019.111558

\hypertarget{interpreting-annual-time-series-with-landtrendr}{%
\section{Interpreting Annual Time Series with
LandTrendr}\label{interpreting-annual-time-series-with-landtrendr}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-10}{%
\subsubsection*{Author}\label{author-10}}

Robert Kennedy, Justin Braaten, Peter Clary

\hypertarget{overview-12}{%
\subsubsection*{Overview}\label{overview-12}}

Time-series analysis of change can be achieved by fitting the entire
spectral trajectory using simple statistical models. These allow us to
both simplify the time series and to extract useful information about
the changes occurring. In this chapter, you will get an introduction to
the use of LandTrendr, one of these time-series approaches used to
characterize time series of spectral values.

\hypertarget{learning-outcomes-12}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-12}}

\begin{itemize}
\tightlist
\item
  Evaluating yearly time-series spectral values to distinguish between
  true change and artifacts.
\item
  Recognizing disturbance and growth signals in the time series of
  annual spectral values for individual pixels.
\item
  Interpreting change segments and translating them to maps.
\item
  Applying parameters in a graphical user interface to create
  disturbance maps in forests.
\end{itemize}

\hypertarget{assumes-you-know-how-to-12}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-12}}

\begin{itemize}
\tightlist
\item
  Calculate and interpret vegetation indices (Chap. F2.0)
\item
  Interpret bands and indices in terms of land surface characteristics
  (Chap. F2.0).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-10}{%
\subsection*{Introduction}\label{introduction-10}}

Land surface change happens all the time, and satellite sensors witness
it. If a spectral index is chosen to match the type of change being
sought, surface change can be inferred from changes in spectral index
values. Over time, the progression of spectral values witnessed in each
pixel tells a story of the processes of change, such as growth and
disturbance. Time-series algorithms are designed to leverage many
observations of spectral values over time to isolate and describe
changes of interest, while ignoring uninteresting change or noise.

In this lab, we use the LandTrendr time-series algorithms to map change.
The LandTrendr algorithms apply ``temporal segmentation'' strategies to
distill a multiyear time series into sequential straight-line segments
that describe the change processes occurring in each pixel. We then
isolate the segment of interest in each pixel and make maps of when, how
long, and how intensely each process occurred. Similar strategies can be
applied to more complicated descriptions of the time series, as is seen
in some of the chapters that follow this one.

For this lab, we will use a graphical user interface (GUI) to teach the
concepts of LandTrendr.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F45a. The book's repository contains information about
accessing the LandTrendr interface.

\end{tcolorbox}

\hypertarget{pixel-time-series}{%
\subsection{Pixel Time Series}\label{pixel-time-series}}

When working with LandTrendr for the first time in your area, there are
two questions you must address.

First, is the change of interest detectable in the spectral reflectance
record? If the change you are interested in does not leave a pattern in
the spectral reflectance record, then an algorithm will not be able to
find it.

Second, can you identify fitting parameters that allow the algorithm to
capture that record? Time series algorithms apply rules to a temporal
sequence of spectral values in a pixel, and simplify the many
observations into more digestible forms, such as the linear segments we
will work with using LandTrendr. The algorithms that do the
simplification are often guided by parameters that control the way the
algorithm does its job.

The best way to begin assessing these questions is to look at the time
series of individual pixels. In Earth Engine, open and run the script
that generates the GUI we have developed to easily deploy the LandTrendr
algorithms. Run the script, and you should see an interface that looks
like the one shown in Fig. 4.5.1.

\begin{figure}

{\centering \includegraphics{./F4/image29.png}

}

\caption{Fig. 4.5.1 The LandTrendr GUI interface, with the control panel
on the left, the Map panel in the center, and the reporting panel on the
right}

\end{figure}

The LandTrendr GUI consists of three panels: a control panel on the
left, a reporting panel on the right, and a Map panel in the center. The
control panel is where all of the functionality of the interface
resides. There are several modules,and each is accessed by clicking on
the double arrow to the right of the title. The Map panel defaults to a
location in Oregon but can be manually moved anywhere in the world. The
reporting panel shows messages about how to use functions, as well as
providing graphical outputs.

Next, expand the ``Pixel Time Series Options'' function. For now, simply
use your mouse to click somewhere on the map. Wait a few seconds even
though it looks like nothing is happening -- be patient!! The GUI has
sent information to Earth Engine to run the LandTrendr algorithms at the
location you have clicked, and is waiting for the results. Eventually
you should see a chart appear in the reporting panel on the right. Fig.
4.5.2 shows what one pixel looks like in an area where the forest burned
and began regrowth. Your chart will probably look different.

\begin{figure}

{\centering \includegraphics{./F4/image88.png}

}

\caption{Fig. 4.5.2 A typical trajectory for a single pixel. The x-axis
shows the year, the y-axis the spectral index value, and the title the
index chosen. The gray line represents the original spectral values
observed by Landsat, and the red line the result of the LandTrendr
temporal segmentation algorithms.}

\end{figure}

The key to success with the LandTrendr algorithm is interpreting these
time series. First, let's examine the components of the chart. The
x-axis shows the year of observation. With LandTrendr, only one
observation per year is used to describe the history of a pixel; later,
we will cover how you control that value. The y-axis shows the spectral
value of the index that is chosen. In the default mode, the Normalized
Burn Ratio (as described in Chap. F4.4). Note that you also have the
ability to pick more indices using the checkboxes on the control panel
on the left. Note that we scale floating point (decimal) indices by
1000. Thus, an NBR value of 1.0 would be displayed as 1000.

In the chart area, the thick gray line represents the spectral values
observed by the satellite for the period of the year selected for a
single 30 m Landsat pixel at the location you have chosen. The red line
is the output from the temporal segmentation that is the heart of the
LandTrendr algorithms. The title of the chart shows the spectral index,
as well as the root-mean-square error of the fit.

To interpret the time series, first know which way is ``up'' and
``down'' for the spectral index you're interested in. For the NBR, the
index goes up in value when there is more vegetation and less soil in a
pixel. It goes down when there is less vegetation. For vegetation
disturbance monitoring, this is useful.

Next, translate that change into the changes of interest for the change
processes you're interested in. For conifer forest systems, the NBR is
useful because it drops precipitously when a disturbance occurs, and it
rises as vegetation grows.

In the case of Fig. 4.5.2, we interpret the abrupt drop as a
disturbance, and the subsequent rise of the index as regrowth or
recovery (though not necessarily to the same type of vegetation).

\begin{figure}

{\centering \includegraphics{./F4/image71.png}

}

\caption{Fig. 4.5.3 For the trajectory in Fig. 4.5.2, we can identify a
segment capturing disturbance based on its abrupt drop in the NBR index,
and the subsequent vegetative recovery}

\end{figure}

Tip: LandTrendr is able to accept any index, and advanced users are
welcome to use indices of their own design. An important consideration
is knowing which direction indicates ``recovery'' and ``disturbance''
for the topic you are interested in. The algorithms favor detection of
disturbance and can be controlled to constrain how quickly recovery is
assumed to occur (see parameters below).

For LandTrendr to have any hope of finding the change of interest, that
change must be manifested in the gray line showing the original spectral
values. If you know that some process is occurring and it is not evident
in the gray line, what can you do?

One option is to change the index. Any single index is simply one view
of the larger spectral space of the Landsat Thematic Mapper sensors. The
change you are interested in may cause spectral change in a different
direction than that captured with some indices. Try choosing different
indices from the list. If you click on different checkboxes and
re-submit the pixel, the fits for all of the different indices will
appear.

Another option is to change the date range. LandTrendr uses one value
per year, but the value that is chosen can be controlled by the user.
It's possible that the change of interest is better identified in some
seasons than others. We use a medoid image compositing approach, which
picks the best single observation each year from a date range of images
in an ImageCollection. In the GUI, you can change the date range of
imagery used for compositing in the Image Collection portion of the
LandTrendr Options menu (Fig. F4.5.4).

\begin{figure}

{\centering \includegraphics{./F4/image7.png}

}

\caption{Fig. 4.5.4 The LandTrendr options menu. Users control the year
and date range in the Image Collection section, the index used for
temporal segmentation in the middle section, and the parameters
controlling the temporal segmentation in the bottom section}

\end{figure}

Change the Start Date and End Date to find a time of year when the
distinction between cover conditions before and during the change
process of interest is greatest.

There are other considerations to keep in mind. First, seasonality of
vegetation, water, or snow often can affect the signal of the change of
interest. And because we use an ImageCollection that spans a range of
dates, it's best to choose a date range where there is not likely to be
a substantial change in vegetative state from the beginning to the end
of the date range. Clouds can be a factor too. Some seasons will have
more cloudiness, which can make it difficult to find good images. Often
with optical sensors, we are constrained to working with periods where
clouds are less prevalent, or using wide date ranges to provide many
opportunities for a pixel to be cloud-free.

It is possible that no combination of index or data range is sensitive
to the change of interest. If that is the case, there are two options:
try using a different sensor and change detection technique, or accept
that the change is not discernible. This can often occur if the change
of interest occupies a small portion of a given 30 m by 30 m Landsat
pixel, or if the spectral manifestation of the change is so subtle that
it is not spectrally separable from non-changed pixels

Even if you as a human can identify the change of interest in the
spectral trajectory of the gray line, an algorithm may not be able to
similarly track it. To give the algorithm a fighting chance, you need to
explore whether different fitting parameters could be used to match the
red fitted line with the gray source image line.

The overall fitting process includes steps to reduce noise and best
identify the underlying signal. The temporal segmentation algorithms are
controlled by fitting parameters that are described in detail in
\href{https://www.google.com/url?q=https://www.zotero.org/google-docs/?Siyubi\&sa=D\&source=editors\&ust=1671458868286041\&usg=AOvVaw1IKLEXLb8EIQLL40vCpog-}{Kennedy
et al.~(2010)}. You adjust these parameters using the Fitting Parameters
block of the LandTrendr Options menu. Below is a brief overview of what
values are often useful, but these will likely change as you use
different spectral indices.

First, the minimum observations needed criterion is used to evaluate
whether a given trajectory has enough unfiltered (i.e., clear
observation) years to run the fitting. We suggest leaving this at the
default of 6.

The segmentation begins with a noise-dampening step to remove spikes
that could be caused by unfiltered clouds or shadows. The spike
threshold parameter controls the degree of filtering. A value of 1.0
corresponds to no filtering, and lower values corresponding to more
severe filtering. We suggest leaving this at 0.9; if changed, a range
from 0.7 to 1.0 is appropriate.

The next step is finding vertices. This begins with the start and end
year as vertex years, progressively adding candidate vertex years based
on deviation from linear fits. To avoid getting an overabundance of
vertex years initially found using this method, we suggest leaving the
vertex count overshoot at a value of 3. A second set of algorithms uses
deflection angle to cull back this overabundance to a set number of
maximum candidate vertex years.

That number of vertex years is controlled by the max\_segments
parameter. As a general rule, your number of segments should be no more
than one-third of the total number of likely yearly observations. The
years of these vertices (X-values) are then passed to the model-building
step. Assuming you are using at least 30 years of the archive, and your
area has reasonable availability of images, a value of 8 is a good
starting point.

In the model-building step, straight-line segments are built by fitting
Y-values (spectral values) for the periods defined by the vertex years
(X-values). The process moves from left to right---early years to late
years. Regressions of each subsequent segment are connected to the end
of the prior segment. Regressions are also constrained to prevent
unrealistic recovery after disturbance, as controlled by the recovery
threshold parameter. A lower value indicates greater constraint: a value
of 1.0 means the constraint is turned off; a value of 0.25 means that
segments that fully recover in faster than four years (4 = 1/0.25) are
not permitted. Note: This parameter has strong control on the fitting,
and is one of the first to explore when testing parameters.
Additionally, the preventOneYearRecovery will disallow fits that have
one-year-duration recovery segments. This may be useful to prevent
overfitting of noisy data in environments where such quick vegetative
recovery is not ecologically realistic.

Once a model of the maximum number of segments is found, successively
simpler models are made by iteratively removing the least informative
vertex. Each model is scored using a pseudo-f statistic, which penalizes
models with more segments, to create a pseudo p-value for each model.
The p-value threshold parameter is used to identify all fits that are
deemed good enough. Start with a value of 0.05, but check to see if the
fitted line appears to capture the salient shape and features of the
gray source trajectory. If you see temporal patterns in the gray line
that are likely not noise (based on your understanding of the system
under study), consider switching the p-value threshold to 0.10 or even
0.15.

Note: because of temporal autocorrelation, these cannot be interpreted
as true f- and p-values, but rather as relative scalars to distinguish
goodness of fit among models. If no good models can be found using these
criteria based on the p-value parameter set by the user, a second
approach is used to solve for the Y-value of all vertex years
simultaneously. If no good model is found, then a straight-line mean
value model is used.

From the models that pass the p-value threshold, one is chosen as the
final fit. It may be the one with the lowest p-value. However, an
adjustment is made to allow more complicated models (those with more
segments) to be picked even if their p-value is within a defined
proportion of the best-scoring model. That proportion is set by the best
model proportion parameter. As an example, a best model proportion value
of 0.75 would allow a more complicated model to be chosen if its score
were greater than 75\% that of the best model.

\hypertarget{translating-pixels-to-maps}{%
\subsection{Translating Pixels to
Maps}\label{translating-pixels-to-maps}}

Although the full time series is the best description of each pixel's
``life history,'' we typically are interested in the behavior of all of
the pixels in our study area. It would be both inefficient to manually
visualize all of them and ineffective to try to summarize areas and
locations. Thus, we seek to make maps.

There are three post-processing steps to convert a segmented trajectory
to a map. First, we identify segments of interest; if we are interested
in disturbance, we find segments whose spectral change indicates loss.
Second, we filter out segments of that type that do not meet criteria of
interest. For example, very low magnitude disturbances can occur when
the algorithm mistakenly finds a pattern in the random noise of the
signal, and thus we do not want to include it. Third, we extract from
the segment of interest something about its character to map on a
pixel-by-pixel basis: its start year, duration, spectral value, or the
value of the spectral change.

Theory: We'll start with a single pixel to learn how to Interpret a
disturbance pixel time series in terms of the dominant disturbance
segment. For the disturbance time series we have used in figures above,
we can identify the key parameters of the segment associated with the
disturbance. For the example above, we have extracted the actual NBR
values of the fitted time series and noted them in a table (Fig. 4.5.5).
This is not part of the GUI -- it is simply used here to work through
the concepts.

\begin{figure}

{\centering \includegraphics{./F4/image84.png}

}

\caption{Fig. 4.5.5 Tracking actual values of fitted trajectories to
learn how we focus on quantification of disturbance. Because we know
that the NBR index drops when vegetation is lost and soil exposure is
increased, we know that a precipitous drop suggests an abrupt loss of
vegetation. Although some early segments show very subtle change, only
the segment between vertex 4 and 5 shows large-magnitude vegetation
loss.}

\end{figure}

From the table shown in Fig. 4.5.5, we can infer several key things
about this pixel:

\begin{itemize}
\tightlist
\item
  It was likely disturbed between 2006 and 2007. This is because the NBR
  value drops precipitously in the segment bounded by vertices
  (breakpoints) in 2006 and 2007.\\
\item
  The magnitude of spectral change was large: 1175 scaled NBR units out
  of a possible range of 2000 scaled units.
\item
  There were small drops in NBR earlier, which may indicate some subtle
  loss of vegetation over a long period in the pixel. These drops,
  however, would need to be explored in a separate analysis because of
  their subtle nature.
\item
  The main disturbance had a disturbance duration of just one year. This
  abruptness combined with the high magnitude suggests a major
  vegetative disturbance such as a harvest or a fire.
\item
  The disturbance was then followed by recovery of vegetation, but not
  to the level before the disturbance. Note: Ecologists will recognize
  the growth signal as one of succession, or active revegetation by
  human intervention.
\end{itemize}

Following the three post-processing steps noted in the introduction to
this section, to map the year of disturbance for this pixel we would
first identify the potential disturbance segments as those with negative
NBR. Then we would hone in on the disturbance of interest by filtering
out potential disturbance segments that are not abrupt and/or of small
magnitude. This would leave only the high-magnitude, short-duration
segment. For that segment, the first year that we have evidence of
disturbance is the first year after the start of the segment. The
segment starts in 2006, which means that 2007 is the first year we have
such evidence. Thus, we would assign 2007 to this pixel.

If we wanted to map the magnitude of the disturbance, we would follow
the same first two steps, but then report for the pixel value the
magnitude difference between the starting and ending segment.

The LandTrendr GUI provides a set of tools to easily apply the same
logic rules to all pixels of interest and create maps. Click on the
Change Filter Options menu. The interface shown in Fig. 4.5.6 appears.

\begin{figure}

{\centering \includegraphics{./F4/image49.png}

}

\caption{Fig. 4.5.6 The menu used to post-process disturbance
trajectories into maps. Select vegetation change type and sort to hone
in on the segment type of interest, then check boxes to apply selective
filters to eliminate uninteresting changes.}

\end{figure}

The first two sections are used to identify the segments of interest.

Select Vegetation Change Type offers the options of gain or loss, which
refer to gain or loss of vegetation, with disturbance assumed to be
related to loss of vegetation. Note: Advanced users can look in the
landtrendr.js library in the ``calcindex'' function to add new indices
with gain and loss defined as they choose. The underlying algorithm is
built to find disturbance in indices that increase when disturbance
occurs, so indices such as NBR or NDVI need to be multiplied by (−1)
before being fed to the LandTrendr algorithm. This is handled in the
calcIndex function.

Select Vegetation Change Sort offers various options that allow you to
choose the segment of interest based on timing or duration. By default,
the greatest magnitude disturbance is chosen.

Each filter (magnitude, duration, etc.) is used to further winnow the
possible segments of interest. All other filters are applied at the
pixel scale, but Filter by MMU is applied to groups of pixels based on a
given minimum mapping unit (MMU). Once all other filters have been
defined, some pixels are flagged as being of interest and others are
not. The MMU filter looks to see how many connected pixels have been
flagged as occurring in the same year, and omits groups smaller in pixel
count than the number indicated here (which defaults to 11 pixels, or
approximately 1 hectare).

If you're following along and making changes, or if you're just using
the default location and parameters, click the Add Filtered Disturbance
Imagery to add this to the map. You should see something like Fig.
4.5.7.

\begin{figure}

{\centering \includegraphics{./F4/image16.png}

}

\caption{Fig. 4.5.7 The basic output from a disturbance mapping
exercise}

\end{figure}

There are multiple layers of disturbance added to the map. Use the map
layers checkboxes to change which is shown. Magnitude of disturbance,
for example, is a map of the delta change between beginning and
endpoints of the segments (Fig. 4.5.8).

\begin{figure}

{\centering \includegraphics{./F4/image14.png}

}

\caption{Fig. 4.5.8 Magnitude of change for the same area}

\end{figure}

\hypertarget{conclusion-12}{%
\subsection*{Conclusion}\label{conclusion-12}}
\addcontentsline{toc}{subsection}{Conclusion}

This exercise provides a baseline sense of how the LandTrendr algorithm
works. The key points are learning how to interpret change in spectral
values in terms of the processes occurring on the ground, and then
translating those into maps.

You can export the images you've made here using Download Options. Links
to materials are available in the chapter checkpoints and LandTrendr
documentation about both the GUI and the script-based versions of the
algorithm. In particular, there are scripts that handle different
components of the fitting and mapping process, and that allow you to
keep track of the fitting and image selection criteria.

\hypertarget{references-8}{%
\subsection*{References}\label{references-8}}
\addcontentsline{toc}{subsection}{References}

Kennedy RE, Yang Z, Cohen WB (2010) Detecting trends in forest
disturbance and recovery using yearly Landsat time series: 1. LandTrendr
- Temporal segmentation algorithms. Remote Sens Environ 114:2897--2910.
https://doi.org/10.1016/j.rse.2010.07.008

Kennedy RE, Yang Z, Gorelick N, et al (2018) Implementation of the
LandTrendr algorithm on Google Earth Engine. Remote Sens 10:691.
https://doi.org/10.3390/rs10050691

\hypertarget{fitting-functions-to-time-series}{%
\section{Fitting Functions to Time
Series}\label{fitting-functions-to-time-series}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-11}{%
\subsubsection*{Author}\label{author-11}}

Andréa Puzzi Nicolau, Karen Dyson, Biplov Bhandari, David Saah, Nicholas
Clinton

\hypertarget{overview-13}{%
\subsubsection*{Overview}\label{overview-13}}

The purpose of this chapter is to establish a foundation for time-series
analysis of remotely sensed data, which is typically arranged as an
ordered stack of images. You will be introduced to the concepts of
graphing time series, using linear modeling to detrend time series, and
fitting harmonic models to time-series data. At the completion of this
chapter, you will be able to perform analysis of multi-temporal data for
determining trend and seasonality on a per-pixel basis.

\hypertarget{learning-outcomes-13}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-13}}

\begin{itemize}
\tightlist
\item
  Graphing satellite imagery values across a time series.
\item
  Quantifying and potentially removing linear trends in time series.
\item
  Fitting linear and harmonic models to individual pixels in time-series
  data.
\end{itemize}

\hypertarget{assumes-you-know-how-to-13}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-13}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\item
  \hspace{0pt}\hspace{0pt}Create a graph using ui.Chart (Chap. F1.3).
\item
  Use normalizedDifference to calculate vegetation indices (Chap. F2.0).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Mask cloud, cloud shadow, snow/ice, and other undesired pixels (Chap.
  F4.3).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-11}{%
\subsection*{Introduction}\label{introduction-11}}

Many natural and man-made phenomena exhibit important annual,
interannual, or longer-term trends that recur---that is, they occur at
roughly regular intervals. Examples include seasonality in leaf patterns
in deciduous forests and seasonal crop growth patterns. Over time,
indices such as the Normalized Difference Vegetation Index (NDVI) will
show regular increases (e.g., leaf-on, crop growth) and decreases (e.g.,
leaf-off, crop senescence), and typically have a long-term, if noisy,
trend such as a gradual increase in NDVI value as an area recovers from
a disturbance.

Earth Engine supports the ability to do complex linear and non-linear
regressions of values in each pixel of a study area. Simple linear
regressions of indices can reveal linear trends that can span multiple
years. Meanwhile, harmonic terms can be used to fit a sine-wave-like
curve. Once you have the ability to fit these functions to time series,
you can answer many important questions. For example, you can define
vegetation dynamics over multiple time scales, identify phenology and
track changes year to year, and identify deviations from the expected
patterns (Bradley et al.~2007, Bullock et al.~2020). There are multiple
applications for these analyses. For example, algorithms to detect
deviations from the expected pattern can be used to identify disturbance
events, including deforestation and forest degradation (Bullock et
al.~2020).

If you have not already done so, be sure to add the book's code
repository to the Code Editor by entering
\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dprojects/gee-edu/book\&sa=D\&source=editors\&ust=1671458868327546\&usg=AOvVaw0m0oT1feMKQKRq3rtuOxfY}{https://code.earthengine.google.com/?accept\_repo=projects/gee-edu/book}
into your browser. The book's scripts will then be available in the
script manager panel.

\hypertarget{multi-temporal-data-in-earth-engine}{%
\subsection{Multi-Temporal Data in Earth
Engine}\label{multi-temporal-data-in-earth-engine}}

As explained in Chaps. F4.0 and F4.1, a time series in Earth Engine is
typically represented as an ImageCollection. Because of image overlaps,
cloud treatments, and filtering choices, an ImageCollection can have any
of the following complex characteristics:

\begin{itemize}
\tightlist
\item
  At each pixel, there might be a distinct number of observations taken
  from a unique set of dates.
\item
  The size (length) of the time series can vary across pixels.
\item
  Data may be missing in any pixel at any point in the sequence (e.g.,
  due to cloud masking).
\end{itemize}

The use of multi-temporal data in Earth Engine introduces two
mind-bending concepts, which we will describe below.

Per-pixel curve fitting. As you have likely encountered in many
settings, a function can be fit through a series of values. In the most
familiar example, a function of the form y = mx + b can represent a
linear trend in data of all kinds. Fitting a straight ``curve'' with
linear regression techniques involves estimating m and b for a set of x
and y values. In a time series, x typically represents time, while y
values represent observations at specific times. This chapter introduces
how to estimate m and b for computed indices through time to model a
potential linear trend in a time series. We then demonstrate how to fit
a sinusoidal wave, which is useful for modeling rising and falling
values, such as NDVI over a growing season. What can be particularly
mind-bending in this setting is the fact that when Earth Engine is asked
to estimate values across a large area, it will fit a function in every
pixel of the study area. Each pixel, then, has its own m and b values,
determined by the number of observations in that pixel, the observed
values, and the dates for which they were observed.

Higher-dimension band values: array images. That more complex conception
of the potential information contained in a single pixel can be
represented in a higher-order Earth Engine structure: the array image.
As you will encounter in this lab, it is possible for a single pixel in
a single band of a single image to contain more than one value. If you
choose to implement an array image, a single pixel might contain a
one-dimensional vector of numbers, perhaps holding the slope and
intercept values resulting from a linear regression, for example. Other
examples, outside the scope of this chapter but used in the next
chapter, might employ a two-dimensional matrix of values for each pixel
within a single band of an image. Higher-order dimensions are available,
as well as array image manipulations borrowed from the world of matrix
algebra. Additionally, there are functions to move between the
multidimensional array image structure and the more familiar, more
easily displayed, simple Image type. Some of these array image functions
were encountered in Chap. F3.1, but with less explanatory context.

First, we will give some very basic notation (Fig. F4.6.1). A scalar
pixel at time t is given by pt, and a pixel vector by pt.~A variable
with a ``hat'' represents an estimated value: in this context, p̂t is the
estimated pixel value at time t. A time series is a collection of pixel
values, usually sorted chronologically: \{pt; t = t0\ldots tN\}, where t
might be in any units, t0 is the smallest, and tN is the largest such t
in the series.

\begin{figure}

{\centering \includegraphics{./F4/image38.png}

}

\caption{Fig. F4.6.1 Time series representation of pixel p}

\end{figure}

\hypertarget{data-preparation-and-preprocessing}{%
\subsection{Data Preparation and
Preprocessing}\label{data-preparation-and-preprocessing}}

The first step in analysis of time-series data is to import data of
interest and plot it at an interesting location. We will work with the
USGS Landsat 8 Level 2, Collection 2, Tier 1 ImageCollection and a
cloud-masking function (Chap. F4.3), scale the image values, and add
variables of interest to the collection as bands. Copy and paste the
code below to filter the Landsat 8 collection to a point of interest
over California (variable roi) and specific dates, and to apply the
defined function. The variables of interest added by the function are:
(1) NDVI (Chap. F2.0), (2) a time variable that is the difference
between the image's current year and the year 1970 (a start point), and
(3) a constant variable with value 1.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{///////////////////// Sections 1 \& 2 /////////////////////////////  }
  
\CommentTok{// Define function to mask clouds, scale, and add variables  }
\CommentTok{// (NDVI, time and a constant) to Landsat 8 imagery.  }
\KeywordTok{function} \FunctionTok{maskScaleAndAddVariable}\NormalTok{(image) \{   }\CommentTok{// Bit 0 {-} Fill   // Bit 1 {-} Dilated Cloud   // Bit 2 {-} Cirrus   // Bit 3 {-} Cloud   // Bit 4 {-} Cloud Shadow   var qaMask = image.select(\textquotesingle{}QA\_PIXEL\textquotesingle{}).bitwiseAnd(parseInt(\textquotesingle{}11111\textquotesingle{},       2)).eq(0);   var saturationMask = image.select(\textquotesingle{}QA\_RADSAT\textquotesingle{}).eq(0);   // Apply the scaling factors to the appropriate bands.   var opticalBands = image.select(\textquotesingle{}SR\_B.\textquotesingle{}).multiply(0.0000275).add({-}       0.2);   var thermalBands = image.select(\textquotesingle{}ST\_B.*\textquotesingle{}).multiply(0.00341802)  }
       \OperatorTok{.}\FunctionTok{add}\NormalTok{(}\FloatTok{149.0}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Replace the original bands with the scaled ones and apply the masks.   var img = image.addBands(opticalBands, null, true)  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(thermalBands}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(qaMask)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(saturationMask)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ imgScaled }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(img}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Now we start to add variables of interest.   // Compute time in fractional years since the epoch.   var date = ee.Date(image.get(\textquotesingle{}system:time\_start\textquotesingle{}));   var years = date.difference(ee.Date(\textquotesingle{}1970{-}01{-}01\textquotesingle{}), \textquotesingle{}year\textquotesingle{});   // Return the image with the added bands.   return imgScaled       // Add an NDVI band.       .addBands(imgScaled.normalizedDifference([\textquotesingle{}SR\_B5\textquotesingle{}, \textquotesingle{}SR\_B4\textquotesingle{}])  }
           \OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{))       }\CommentTok{// Add a time band.       .addBands(ee.Image(years).rename(\textquotesingle{}t\textquotesingle{}))  }
       \OperatorTok{.}\FunctionTok{float}\NormalTok{()       }\CommentTok{// Add a constant band.       .addBands(ee.Image.constant(1));  }
\NormalTok{\}  }
  
\CommentTok{// Import point of interest over California, USA.  }
\KeywordTok{var}\NormalTok{ roi }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{121.059}\OperatorTok{,} \FloatTok{37.9242}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Import the USGS Landsat 8 Level 2, Collection 2, Tier 1 image collection),  }
\CommentTok{// filter, mask clouds, scale, and add variables.  }
\KeywordTok{var}\NormalTok{ landsat8sr }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(roi)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2013{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2018{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(maskScaleAndAddVariable)}\OperatorTok{;}  
  
\CommentTok{// Set map center over the ROI.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(roi}\OperatorTok{,} \DecValTok{6}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, to visualize the NDVI at the point of interest over time, copy and
paste the code below to print a chart of the time series (Chap. F1.3) at
the location of interest (Fig. F4.6.2).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Plot a time series of NDVI at a single location.  }
\KeywordTok{var}\NormalTok{ landsat8Chart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(landsat8sr}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{)}\OperatorTok{,}\NormalTok{ roi)  }
   \OperatorTok{.}\FunctionTok{setChartType}\NormalTok{(}\StringTok{\textquotesingle{}ScatterChart\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Landsat 8 NDVI time series at ROI\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{lineWidth}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{pointSize}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
\NormalTok{   \})}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(landsat8Chart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image3.png}

}

\caption{Fig. F4.6.2 Time series representation of pixel p}

\end{figure}

We can add a linear trend line to our chart using the trendlines
parameters in the setOptions function for image series charts. Copy and
paste the code below to print the same chart but with a linear trend
line plotted (Fig. F4.6.3). In the next section, you will learn how to
estimate linear trends over time.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Plot a time series of NDVI with a linear trend line  }
\CommentTok{// at a single location.  }
\KeywordTok{var}\NormalTok{ landsat8ChartTL }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(landsat8sr}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{)}\OperatorTok{,}\NormalTok{ roi)  }
   \OperatorTok{.}\FunctionTok{setChartType}\NormalTok{(}\StringTok{\textquotesingle{}ScatterChart\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Landsat 8 NDVI time series at ROI\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{trendlines}\OperatorTok{:}\NormalTok{ \{           }\DecValTok{0}\OperatorTok{:}\NormalTok{ \{  }
               \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}CC0000\textquotesingle{}}\NormalTok{           \}  }
\NormalTok{       \}}\OperatorTok{,}  
       \DataTypeTok{lineWidth}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{pointSize}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
\NormalTok{   \})}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(landsat8ChartTL)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image82.png}

}

\caption{Fig. F4.6.3 Time series representation of pixel p with the
trend line in red}

\end{figure}

Now that we have plotted and visualized the data, lots of interesting
analyses can be done to the time series by harnessing Earth Engine tools
for fitting curves through this data. We will see a couple of examples
in the following sections.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F46a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{estimating-linear-trend-over-time}{%
\subsection{Estimating Linear Trend Over
Time}\label{estimating-linear-trend-over-time}}

Time series datasets may contain not only trends but also seasonality,
both of which may need to be removed prior to modeling. Trends and
seasonality can result in a varying mean and a varying variance over
time, both of which define a time series as non-stationary. Stationary
datasets, on the other hand, have a stable mean and variance, and are
therefore much easier to model.

Consider the following linear model, where et is a random error:

pt = β0 + β1t + et (Eq. F4.6.1)

This is the model behind the trend line added to the chart created in
the previous section (Fig. F4.6.3). Identifying trends at different
scales is a big topic, with many approaches being used (e.g.,
differencing, modeling).

Removing unwanted to uninteresting trends for a given problem is often a
first step to understanding complex patterns in time series. There are
several approaches to remove trends. Here, we will remove the linear
trend that is evident in the data shown in Fig. F4.6.3 using Earth
Engine's built-in tools for regression modeling. This approach is a
useful, straightforward way to detrend data in time series (Shumway and
Stoffer 2019). Here, the goal is to discover the values of the β's in
Eq. F4.6.1 for each pixel.

Copy and paste code below into the Code Editor, adding it to the end of
the script from the previous section. Running this code will fit this
trend model to the Landsat-based NDVI series using ordinary least
squares, using the linearRegression reducer (Chap. F3.0).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{///////////////////// Section 3 /////////////////////////////  }
  
\CommentTok{// List of the independent variable names  }
\KeywordTok{var}\NormalTok{ independents }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{List}\NormalTok{([}\StringTok{\textquotesingle{}constant\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}t\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Name of the dependent variable.  }
\KeywordTok{var}\NormalTok{ dependent }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{String}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Compute a linear trend.  This will have two bands: \textquotesingle{}residuals\textquotesingle{} and  }
\CommentTok{// a 2x1 (Array Image) band called \textquotesingle{}coefficients\textquotesingle{}.  }
\CommentTok{// (Columns are for dependent variables)  }
\KeywordTok{var}\NormalTok{ trend }\OperatorTok{=}\NormalTok{ landsat8sr}\OperatorTok{.}\FunctionTok{select}\NormalTok{(independents}\OperatorTok{.}\FunctionTok{add}\NormalTok{(dependent))  }
   \OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{linearRegression}\NormalTok{(independents}\OperatorTok{.}\FunctionTok{length}\NormalTok{()}\OperatorTok{,} \DecValTok{1}\NormalTok{))}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(trend}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}trend array image\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Flatten the coefficients into a 2{-}band image.  }
\KeywordTok{var}\NormalTok{ coefficients }\OperatorTok{=}\NormalTok{ trend}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}coefficients\textquotesingle{}}\NormalTok{)   }\CommentTok{// Get rid of extra dimensions and convert back to a regular image   .arrayProject([0])  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([independents])}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(coefficients}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}coefficients image\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If you click over a point using the Inspector tab, you will see the
pixel values for the array image (coefficients ``t'' and ``constant'',
and residuals) and two-band image (coefficients ``t'' and ``constant'')
(Fig. F4.6.4).

\begin{figure}

{\centering \includegraphics{./F4/image53.png}

}

\caption{Fig. F4.6.4 Pixel values of array image and coefficients image}

\end{figure}

Now, copy and paste the code below to use the model to detrend the
original NDVI time series and plot the time series chart with the
trendlines parameter (Fig. F4.6.5).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Compute a detrended series.  }
\KeywordTok{var}\NormalTok{ detrended }\OperatorTok{=}\NormalTok{ landsat8sr}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{   }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(dependent)}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(  }
\NormalTok{           image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(independents)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(coefficients)  }
           \OperatorTok{.}\FunctionTok{reduce}\NormalTok{(}\StringTok{\textquotesingle{}sum\textquotesingle{}}\NormalTok{))  }
       \OperatorTok{.}\FunctionTok{rename}\NormalTok{(dependent)  }
       \OperatorTok{.}\FunctionTok{copyProperties}\NormalTok{(image}\OperatorTok{,}\NormalTok{ [}\StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Plot the detrended results.  }
\KeywordTok{var}\NormalTok{ detrendedChart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(detrended}\OperatorTok{,}\NormalTok{ roi}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \DecValTok{30}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Detrended Landsat time series at ROI\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{lineWidth}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{pointSize}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
       \DataTypeTok{trendlines}\OperatorTok{:}\NormalTok{ \{           }\DecValTok{0}\OperatorTok{:}\NormalTok{ \{  }
               \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}CC0000\textquotesingle{}}\NormalTok{           \}  }
\NormalTok{       \}}\OperatorTok{,}  
\NormalTok{   \})}\OperatorTok{;}\FunctionTok{print}\NormalTok{(detrendedChart)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image20.png}

}

\caption{Fig. F4.6.5 Detrended NDVI time series}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F46b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{estimating-seasonality-with-a-harmonic-model}{%
\subsection{Estimating Seasonality with a Harmonic
Model}\label{estimating-seasonality-with-a-harmonic-model}}

A linear trend is one of several possible types of trends in time
series. Time series can also present harmonic trends, in which a value
goes up and down in a predictable wave pattern. These are of particular
interest and usefulness in the natural world, where harmonic changes in
greenness of deciduous vegetation can occur across the spring, summer,
and autumn. Now we will return to the initial time series (landsat8sr)
of Fig. F4.6.2 and fit a harmonic pattern through the data. Consider the
following harmonic model, where A is amplitude, ω is frequency, φ is
phase, and et is a random error.

pt = β0 + β1t + Acos(2πωt - φ) + et

\begin{verbatim}
= β0 + β1t + β2cos(2πωt) + β3sin(2πωt) + et                (Eq. F4.6.2)
\end{verbatim}

Note that β2 = Acos(φ) and β3 = Asin(φ), implying A = (β22 + β32)½ and φ
= atan(β3/β2) (as described in Shumway and Stoffer 2019). To fit this
model to an annual time series, set ω = 1 (one cycle per year) and use
ordinary least squares regression.

The setup for fitting the model is to first add the harmonic variables
(the third and fourth terms of Eq. F4.6.2) to the ImageCollection. Then,
fit the model as with the linear trend, using the linearRegression
reducer, which will yield a 4 x 1 array image.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{///////////////////// Section 4 /////////////////////////////  }
  
\CommentTok{// Use these independent variables in the harmonic regression.  }
\KeywordTok{var}\NormalTok{ harmonicIndependents }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{List}\NormalTok{([}\StringTok{\textquotesingle{}constant\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}t\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}cos\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}sin\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Add harmonic terms as new image bands.  }
\KeywordTok{var}\NormalTok{ harmonicLandsat }\OperatorTok{=}\NormalTok{ landsat8sr}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{   }\KeywordTok{var}\NormalTok{ timeRadians }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}t\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\DecValTok{2} \OperatorTok{*} \BuiltInTok{Math}\OperatorTok{.}\ConstantTok{PI}\NormalTok{)}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ image       }\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(timeRadians}\OperatorTok{.}\FunctionTok{cos}\NormalTok{()}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}cos\textquotesingle{}}\NormalTok{))  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(timeRadians}\OperatorTok{.}\FunctionTok{sin}\NormalTok{()}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}sin\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Fit the model.  }
\KeywordTok{var}\NormalTok{ harmonicTrend }\OperatorTok{=}\NormalTok{ harmonicLandsat  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(harmonicIndependents}\OperatorTok{.}\FunctionTok{add}\NormalTok{(dependent))   }\CommentTok{// The output of this reducer is a 4x1 array image.   .reduce(ee.Reducer.linearRegression(harmonicIndependents.length(),       1));}
\end{Highlighting}
\end{Shaded}

Now, copy and paste the code below to plug the coefficients into Eq.
F4.6.2 in order to get a time series of fitted values and plot the
harmonic model time series (Fig. F4.6.6).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Turn the array image into a multi{-}band image of coefficients.  }
\KeywordTok{var}\NormalTok{ harmonicTrendCoefficients }\OperatorTok{=}\NormalTok{ harmonicTrend}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}coefficients\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{arrayProject}\NormalTok{([}\DecValTok{0}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([harmonicIndependents])}\OperatorTok{;}  
  
\CommentTok{// Compute fitted values.  }
\KeywordTok{var}\NormalTok{ fittedHarmonic }\OperatorTok{=}\NormalTok{ harmonicLandsat}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{   }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(  }
\NormalTok{       image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(harmonicIndependents)  }
       \OperatorTok{.}\FunctionTok{multiply}\NormalTok{(harmonicTrendCoefficients)  }
       \OperatorTok{.}\FunctionTok{reduce}\NormalTok{(}\StringTok{\textquotesingle{}sum\textquotesingle{}}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}fitted\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Plot the fitted model and the original data at the ROI.  }
\FunctionTok{print}\NormalTok{(ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(  }
\NormalTok{       fittedHarmonic}\OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}fitted\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{])}\OperatorTok{,}\NormalTok{ roi}\OperatorTok{,}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}  
       \OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,} \DecValTok{30}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{setSeriesNames}\NormalTok{([}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}fitted\textquotesingle{}}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Harmonic model: original and fitted values\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{lineWidth}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{pointSize}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
\NormalTok{   \}))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image86.png}

}

\caption{Fig. F4.6.6 Harmonic model of NDVI time series}

\end{figure}

Returning to the mind-bending nature of curve-fitting, it is worth
remembering that the harmonic waves seen in Fig. F4.6.6 are the fit of
the data to a single point across the image. Next, we will map the
outcomes of millions of these fits, pixel by pixel, across the entire
study area.

We'll compute and map the phase and amplitude of the estimated harmonic
model for each pixel. Phase and amplitude (Fig. F4.6.7) can give us
additional information to facilitate remote sensing applications such as
agricultural mapping and land use and land cover monitoring.
Agricultural crops with different phenological cycles can be
distinguished with phase and amplitude information, something that
perhaps would not be possible with spectral information alone.

\begin{figure}

{\centering \includegraphics{./F4/image8.png}

}

\caption{Fig. F4.6.7 Example of phase and amplitude in harmonic model}

\end{figure}

Copy and paste the code below to compute phase and amplitude from the
coefficients and add this image to the map (Fig. F4.6.8).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Compute phase and amplitude.  }
\KeywordTok{var}\NormalTok{ phase }\OperatorTok{=}\NormalTok{ harmonicTrendCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}sin\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{atan2}\NormalTok{(harmonicTrendCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}cos\textquotesingle{}}\NormalTok{))   }\CommentTok{// Scale to [0, 1] from radians.   .unitScale({-}Math.PI, Math.PI);  }
  
\KeywordTok{var}\NormalTok{ amplitude }\OperatorTok{=}\NormalTok{ harmonicTrendCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}sin\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{hypot}\NormalTok{(harmonicTrendCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}cos\textquotesingle{}}\NormalTok{))   }\CommentTok{// Add a scale factor for visualization.   .multiply(5);  }
  
\CommentTok{// Compute the mean NDVI.  }
\KeywordTok{var}\NormalTok{ meanNdvi }\OperatorTok{=}\NormalTok{ landsat8sr}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Use the HSV to RGB transformation to display phase and amplitude.  }
\KeywordTok{var}\NormalTok{ rgb }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\FunctionTok{cat}\NormalTok{([  }
\NormalTok{   phase}\OperatorTok{,} \CommentTok{// hue   amplitude, // saturation (difference from white)   meanNdvi // value (difference from black)  }
\NormalTok{])}\OperatorTok{.}\FunctionTok{hsvToRgb}\NormalTok{()}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(rgb}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}phase (hue), amplitude (sat), ndvi (val)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image63.png}

}

\caption{Fig. F4.6.8 Phase, amplitude, and NDVI concatenated image}

\end{figure}

The code uses the HSV to RGB transformation hsvToRgb for visualization
purposes (Chap. F3.1). We use this transformation to separate color
components from intensity for a better visualization. Without this
transformation, we would visualize a very colorful image that would not
look as intuitive as the image with the transformation. With this
transformation, phase, amplitude, and mean NDVI are displayed in terms
of hue (color), saturation (difference from white), and value
(difference from black), respectively. Therefore, darker pixels are
areas with low NDVI. For example, water bodies will appear as black,
since NDVI values are zero or negative. The different colors are
distinct phase values, and the saturation of the color refers to the
amplitude: whiter colors mean amplitude closer to zero (e.g., forested
areas), and the more vivid the colors, the higher the amplitude (e.g.,
croplands). Note that if you use the Inspector tool to analyze the
values of a pixel, you will not get values of phase, amplitude, and
NDVI, but the transformed values into values of blue, green, and red
colors.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F46c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{an-application-of-curve-fitting}{%
\subsection{An Application of Curve
Fitting}\label{an-application-of-curve-fitting}}

\hypertarget{the-rich-data-about-the-curve-fits-can-be-viewed-in-a-multitude-of-different-ways.-add-the-code-below-to-your-script-to-produce-the-view-in-fig.-4.6.9.-the-image-will-be-a-close-up-of-the-area-around-modesto-california.}{%
\subsubsection{The rich data about the curve fits can be viewed in a
multitude of different ways. Add the code below to your script to
produce the view in Fig. 4.6.9. The image will be a close-up of the area
around Modesto,
California.}\label{the-rich-data-about-the-curve-fits-can-be-viewed-in-a-multitude-of-different-ways.-add-the-code-below-to-your-script-to-produce-the-view-in-fig.-4.6.9.-the-image-will-be-a-close-up-of-the-area-around-modesto-california.}}

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{///////////////////// Section 5 /////////////////////////////  }
  
\CommentTok{// Import point of interest over California, USA.  }
\KeywordTok{var}\NormalTok{ roi }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{121.04}\OperatorTok{,} \FloatTok{37.641}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Set map center over the ROI.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(roi}\OperatorTok{,} \DecValTok{14}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ trend0D }\OperatorTok{=}\NormalTok{ trend}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}coefficients\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{arrayProject}\NormalTok{([}\DecValTok{0}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([independents])}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}t\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ anotherView }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(harmonicTrendCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}sin\textquotesingle{}}\NormalTok{))  }
   \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(trend0D)  }
   \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(harmonicTrendCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}cos\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(anotherView}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.03}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.03}\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}Another combination of fit characteristics\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\includegraphics{./F4/image61.png}

\begin{figure}

{\centering \includegraphics{./F4/image45.png}

}

\caption{Fig. F4.6.9 Two views of the harmonic fits for NDVI for the
Modesto, California area}

\end{figure}

The upper image in Fig. F4.6.9 is a closer view of Fig. F4.6.8, showing
an image that transforms the sine and cosine coefficient values, and
incorporates information from the mean NDVI. The lower image draws the
sine and cosine in the red and blue bands, and extracts the slope of the
linear trend that you calculated earlier in the chapter, placing that in
the green band. The two views of the fit are similarly structured in
their spatial pattern---both show fields to the west and the city to the
east. But the pixel-by-pixel variability emphasizes a key point of this
chapter: that a fit to the NDVI data is done independently in each pixel
in the image. Using different elements of the fit, these two views, like
other combinations of the data you might imagine, can reveal the rich
variability of the landscape around Modesto.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F46d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{higher-order-harmonic-models}{%
\subsection{Higher-Order Harmonic
Models}\label{higher-order-harmonic-models}}

Harmonic models are not limited to fitting a single wave through a set
of points. In some situations, there may be more than one cycle within a
given year---for example, when an agricultural field is double-cropped.
Modeling multiple waves within a given year can be done by adding more
harmonic terms to Eq. F4.6.2. The code at the following checkpoint
allows the fitting of any number of cycles through a given point.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F46e. The book's repository contains a script to use to
begin this section. You will need to start with that script and edit the
code to produce the charts in this section.

\end{tcolorbox}

Beginning with the repository script, changing the value of the
harmonics variable will change the complexity of the harmonic curve fit
by superimposing more or fewer harmonic waves on each other. While
fitting higher-order functions improves the goodness-of-fit of the model
to a given set of data, many of the coefficients may be close to zero at
higher numbers or harmonic terms. Fig. F4.6.10 shows the fit through the
example point using one, two, and three harmonic curves.

\includegraphics{./F4/image24.png}

\includegraphics{./F4/image35.png}

\begin{figure}

{\centering \includegraphics{./F4/image64.png}

}

\caption{Fig. F4.6.10 Fit with harmonic curves of increasing complexity,
fitted for data at a given point}

\end{figure}

\hypertarget{conclusion-13}{%
\subsection*{Conclusion}\label{conclusion-13}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, we learned how to graph and fit both linear and
harmonic functions to time series of remotely sensed data. These skills
underpin important tools such as Continuous Change Detection and
Classification (CCDC, Chap. F4.7) and Continuous Degradation Detection
(CODED, Chap. A3.4). These approaches are used by many organizations to
detect forest degradation and deforestation (e.g., Tang et al.~2019,
Bullock et al.~2020). These approaches can also be used to identify
crops (Chap. A1.1) with high degrees of accuracy (Ghazaryan et
al.~2018).

\hypertarget{references-9}{%
\subsection*{References}\label{references-9}}
\addcontentsline{toc}{subsection}{References}

Bradley BA, Jacob RW, Hermance JF, Mustard JF (2007) A curve fitting
procedure to derive inter-annual phenologies from time series of noisy
satellite NDVI data. Remote Sens Environ 106:137--145.
https://doi.org/10.1016/j.rse.2006.08.002

Bullock EL, Woodcock CE, Olofsson P (2020) Monitoring tropical forest
degradation using spectral unmixing and Landsat time series analysis.
Remote Sens Environ 238:110968.
https://doi.org/10.1016/j.rse.2018.11.011

Ghazaryan G, Dubovyk O, Löw F, et al (2018) A rule-based approach for
crop identification using multi-temporal and multi-sensor phenological
metrics. Eur J Remote Sens 51:511--524.
https://doi.org/10.1080/22797254.2018.1455540

Shumway RH, Stoffer DS (2019) Time Series: A Data Analysis Approach
Using R. Chapman and Hall/CRC

Tang X, Bullock EL, Olofsson P, et al (2019) Near real-time monitoring
of tropical forest disturbance: New algorithms and assessment framework.
Remote Sens Environ 224:202--218.
https://doi.org/10.1016/j.rse.2019.02.003

\hypertarget{interpreting-time-series-with-ccdc}{%
\section{Interpreting Time Series with
CCDC}\label{interpreting-time-series-with-ccdc}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-12}{%
\subsubsection*{Author}\label{author-12}}

Paulo Arévalo, Pontus Olofsson

\hypertarget{overview-14}{%
\subsubsection*{Overview}\label{overview-14}}

Continuous Change Detection and Classification (CCDC) is a land change
monitoring algorithm designed to operate on time series of satellite
data, particularly Landsat data. This chapter focuses on the portion
that is the change detection component (CCD); you will learn how to run
the algorithm, interpret its outputs, and visualize coefficients and
change information.

\hypertarget{learning-outcomes-14}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-14}}

\begin{itemize}
\tightlist
\item
  Exploring pixel-level time series of Landsat observations, as well as
  the temporal segments that CCDC fits to the observations.
\item
  Visualizing the coefficients of the temporal segments in space.
\item
  Visualizing predicted images made from detected temporal segments.
\item
  Visualizing change information.
\item
  Using array image functions.
\item
  Attaching user-defined metadata to an image when exporting.
\end{itemize}

\hypertarget{assumes-you-know-how-to-14}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-14}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\item
  Visualize images with a variety of false-color band combinations
  (Chap. F1.1).
\item
  Interpret bands and indices in terms of land surface characteristics
  (Chap. F2.0).
\item
  Work with array images (Chap. F3.1, Chap. F4.6).
\item
  Interpret fitted harmonic models (Chap. F4.6).
\end{itemize}

\hypertarget{introduction-to-theory}{%
\subsection{Introduction to Theory}\label{introduction-to-theory}}

``A time series is a sequence of observations taken sequentially in
time. \ldots{} An intrinsic feature of a time series is that, typically,
adjacent observations are dependent. Time-series analysis is concerned
with techniques for the analysis of this dependency.'' This is the
formal definition of time-series analysis by Box et al.~(1994). In a
remote sensing context, the observations of interest are measurements of
radiation reflected from the surface of the Earth from the Sun or an
instrument emitting energy toward Earth. Consecutive measurements made
over a given area result in a time series of surface reflectance. By
analyzing such time series, we can achieve a comprehensive
characterization of ecosystem and land surface processes (Kennedy et
al.~2014). The result is a shift away from traditional, retrospective
change-detection approaches based on data acquired over the same area at
two or a few points in time to continuous monitoring of the landscape
(Woodcock et al.~2020). Previous obstacles related to data storage,
preprocessing, and computing power have been largely overcome with the
emergence of powerful cloud-computing platforms that provide direct
access to the data (Gorelick et al.~2017). In this chapter, we will
illustrate how to study landscape dynamics in the Amazon river basin by
analyzing dense time series of Landsat data using the CCDC algorithm.
Unlike LandTrendr (Chap. F4.5), which uses anniversary images to fit
straight line segments that describe the spectral trajectory over time,
CCDC uses all available clear observations. This has multiple
advantages, including the ability to detect changes within a year and
capture seasonal patterns, although at the expense of much higher
computational demands and more complexity to manipulate the outputs,
compared to LandTrendr.

\hypertarget{understanding-temporal-segmentation-with-ccdc}{%
\subsection{Understanding Temporal Segmentation with
CCDC}\label{understanding-temporal-segmentation-with-ccdc}}

Spectral change is detected at the pixel level by testing for structural
breaks in a time series of reflectance. In Earth Engine, this process is
referred to as ``temporal segmentation,'' as pixel-level time series are
segmented according to periods of unique reflectance. It does so by
fitting harmonic regression models to all spectral bands in the time
series. The model-fitting starts at the beginning of the time series and
moves forward in time in an ``online'' approach to change detection. The
coefficients are used to predict future observations, and if the
residuals of future observations exceed a statistical threshold for
numerous consecutive observations, then the algorithm flags that a
change has occurred. After the change, a new regression model is fit and
the process continues until the end of the time series. The details of
the original algorithm are described in Zhu and Woodcock (2014). We have
created an interface-based tool (Arévalo et al.~2020) that facilitates
the exploration of time series of Landsat observations and the CCDC
results.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F47a. The book's repository contains information about
accessing the CCDC interface.

\end{tcolorbox}

Once you have loaded the CCDC interface (Fig. F4.7.1), you will be able
to navigate to any location, pick a Landsat spectral band or index to
plot, and click on the map to see the fit by CCDC at the location you
clicked. For this exercise, we will study landscape dynamics in the
state of Rondônia, Brazil. We can use the panel on the left-bottom
corner to enter the following coordinates (latitude, longitude):
-9.0002, -62.7223. A point will be added in that location and the map
will zoom in to it. Once there, click on the point and wait for the
chart at the bottom to load. This example shows the Landsat time series
for the first shortwave infrared (SWIR1) band (as blue dots) and the
time segments (as colored lines) run using CCDC default parameters. The
first segment represents stable forest, which was abruptly cut in
mid-2006. The algorithm detects this change event and fits a new segment
afterwards, representing a new temporal pattern of agriculture. Other
subsequent patterns are detected as new segments are fitted that may
correspond to cycles of harvest and regrowth, or a different crop. To
investigate the dynamics over time, you can click on the points in the
chart, and the Landsat images they correspond to will be added to the
map according to the visualization parameters selected for the RGB
combination in the left panel. Currently, changes made in that panel are
not immediate but must be set before clicking on the map.

Pay special attention to the characteristics of each segment. For
example, look at the average surface reflectance value for each segment.
The presence of a pronounced slope may be indicative of phenomena like
vegetation regrowth or degradation. The number of harmonics used in each
segment may represent seasonality in vegetation (either natural or due
to agricultural practices) or landscape dynamics (e.g., seasonal
flooding).

\begin{figure}

{\centering \includegraphics{./F4/image6.png}

}

\caption{Fig. 4.7.1 Landsat time series for the SWIR1 band (blue dots)
and CCDC time segments (colored lines) showing a forest loss event circa
2006 for a place in Rondônia, Brazil}

\end{figure}

Question 1. While still using the SWIR1 band, click on a pixel that is
forested. What do the time series and time segments look like?

\hypertarget{running-ccdc}{%
\subsection{Running CCDC}\label{running-ccdc}}

The tool shown above is useful for understanding the temporal dynamics
for a specific point. However, we can do a similar analysis for larger
areas by first running the CCDC algorithm over a group of pixels. The
CCDC function in Earth Engine can take any ImageCollection, ideally one
with little or no noise, such as a Landsat ImageCollection where clouds
and cloud shadows have been masked. CCDC contains an internal cloud
masking algorithm and is rather robust against missed clouds, but the
cleaner the data the better. To simplify the process, we have developed
a function library that contains functions for generating input data and
processing CCDC results. Paste this line of code in a new script:

var utils = require(
`users/parevalo\_bu/gee-ccdc-tools:ccdcUtilities/api');

For the current exercise, we will obtain an ImageCollection of Landsat
4, 5, 7, and 8 data (Collection 2 Tier 1) that has been filtered for
clouds, cloud shadows, haze, and radiometrically saturated pixels. If we
were to do this manually, we would retrieve each ImageCollection for
each satellite, apply the corresponding filters and then merge them all
into a single ImageCollection. Instead, to simplify that process, we
will use the function getLandsat, included in the ``Inputs'' module of
our utilities, and then filter the resulting ImageCollection to a small
study region for the period between 2000 and 2020. The getLandsat
function will retrieve all surface reflectance bands (renamed and scaled
to actual surface reflectance units) as well as other vegetation
indices. To simplify the exercise, we will select only the surface
reflectance bands we are going to use, adding the following code to your
script:

var studyRegion = ee.Geometry.Rectangle({[}\\
{[}-63.9533, -10.1315{]},\\
{[}-64.9118, -10.6813{]}\\
{]});

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define start, end dates and Landsat bands to use.  }
\KeywordTok{var}\NormalTok{ startDate }\OperatorTok{=} \StringTok{\textquotesingle{}2000{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=} \StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ bands }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}BLUE\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}GREEN\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}RED\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Retrieve all clear, Landsat 4, 5, 7 and 8 observations (Collection 2, Tier 1).  }
\KeywordTok{var}\NormalTok{ filteredLandsat }\OperatorTok{=}\NormalTok{ utils}\OperatorTok{.}\AttributeTok{Inputs}\OperatorTok{.}\FunctionTok{getLandsat}\NormalTok{(\{  }
       \DataTypeTok{collection}\OperatorTok{:} \DecValTok{2}\NormalTok{   \})  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(studyRegion)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(bands)}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(filteredLandsat}\OperatorTok{.}\FunctionTok{first}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

With the ImageCollection ready, we can specify the CCDC parameters and
run the algorithm. For this exercise we will use the default parameters,
which tend to work reasonably well in most circumstances. The only
parameters we will modify are the breakpoint bands, date format, and
lambda. We will set all the parameter values in a dictionary that we
will pass to the CCDC function. For the break detection process we use
all bands except for the blue and surface temperature bands (`BLUE' and
`TEMP', respectively). The minObservations default value of 6 represents
the number of consecutive observations required to flag a change. The
chiSquareProbability and minNumOfYearsScaler default parameters of 0.99
and 1.33, respectively, control the sensitivity of the algorithm to
detect change and the iterative curve fitting process required to detect
change. We set the date format to 1, which corresponds to fractional
years and tends to be easier to interpret. For instance, a change
detected in the middle day of the year 2010 would be stored in a pixel
as 2010.5. Finally, we use the default value of lambda of 20, but we
scale it to match the scale of the inputs (surface reflectance units),
and we specify a maxIterations value of 10000, instead of the default of
25000, which might take longer to complete. Those two parameters control
the curve fitting process.

To complete the input parameters, we specify the ImageCollection to use,
which we derived in the previous code section. Add this code below:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Set CCD params to use.  }
\KeywordTok{var}\NormalTok{ ccdParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{breakpointBands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}GREEN\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}RED\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{tmaskBands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}GREEN\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{minObservations}\OperatorTok{:} \DecValTok{6}\OperatorTok{,}  
   \DataTypeTok{chiSquareProbability}\OperatorTok{:} \FloatTok{0.99}\OperatorTok{,}  
   \DataTypeTok{minNumOfYearsScaler}\OperatorTok{:} \FloatTok{1.33}\OperatorTok{,}  
   \DataTypeTok{dateFormat}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{lambda}\OperatorTok{:} \FloatTok{0.002}\OperatorTok{,}  
   \DataTypeTok{maxIterations}\OperatorTok{:} \DecValTok{10000}\OperatorTok{,}  
   \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ filteredLandsat  }
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Run CCD.  }
\KeywordTok{var}\NormalTok{ ccdResults }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Algorithms}\OperatorTok{.}\AttributeTok{TemporalSegmentation}\OperatorTok{.}\FunctionTok{Ccdc}\NormalTok{(ccdParams)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(ccdResults)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Notice that the output ccdResults contains a large number of bands, with
some of them corresponding to two-dimensional arrays. We will explore
these bands more in the following section. The process of running the
algorithm interactively for more than a handful of pixels can become
very taxing to the system very quickly, resulting in memory errors. To
avoid having such issues, we typically export the results to an Earth
Engine asset first, and then inspect the asset. This approach ensures
that CCDC completes its run successfully, and also allows us to access
the results easily later. In the following sections of this chapter, we
will use a precomputed asset, instead of asking you to export the asset
yourself. For your reference, the code required to export CCDC results
is shown below, with the flag set to false to help you remember to not
export the results now, but instead to use the precomputed asset in the
following sections.

var exportResults = false\\
if (exportResults) \{ // Create a metadata dictionary with the
parameters and arguments used. var metadata = ccdParams;\\
metadata{[}`breakpointBands'{]} =\\
metadata{[}`breakpointBands'{]}.toString();\\
metadata{[}`tmaskBands'{]} = metadata{[}`tmaskBands'{]}.toString();\\
metadata{[}`startDate'{]} = startDate;\\
metadata{[}`endDate'{]} = endDate;\\
metadata{[}`bands'{]} = bands.toString(); // Export results, assigning
the metadata as image properties. // Export.image.toAsset(\{\\
image: ccdResults.set(metadata),\\
region: studyRegion,\\
pyramidingPolicy: \{ ``.default'': `sample' \},\\
scale: 30 \});\\
\}

Note the metadata variable above. This is not strictly required for
exporting the per-pixel CCDC results, but it allows us to keep a record
of important properties of the run by attaching this information as
metadata to the image. Additionally, some of the tools we have created
to interact with CCDC outputs use this user-created metadata to
facilitate using the asset. Note also that setting the value of
pyramidingPolicy to `sample' ensures that all the bands in the output
have the proper policy.

As a general rule, try to use pre-existing CCDC results if possible, and
if you want to try running it yourself outside of this lab exercise,
start with very small areas. For instance, the study area in this
exercise would take approximately 30 minutes on average to export, but
larger tiles may take several hours to complete, depending on the number
of images in the collection and the parameters used.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F47b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{extracting-break-information}{%
\subsection{Extracting Break
Information}\label{extracting-break-information}}

We will now start exploring the pre-exported CCDC results mentioned in
the previous section. We will make use of the third-party module
palettes, described in detail in Chap. F6.0, that simplifies the use of
palettes for visualization. Paste the following code in a new script:

var palettes = require(`users/gena/packages:palettes');

var resultsPath =
`projects/gee-book/assets/F4-7/Rondonia\_example\_small';\\
var ccdResults = ee.Image(resultsPath);\\
Map.centerObject(ccdResults, 10);\\
print(ccdResults);

The first line calls a library that will facilitate visualizing the
images. The second line contains the path to the precomputed results of
the CCDC run shown in the previous section. The printed asset will
contain the following bands:

\begin{itemize}
\tightlist
\item
  tStart: The start date of each time segment
\item
  tEnd: The end date of each time segment
\item
  tBreak: The time segment break date if a change is detected
\item
  numObs: The number of observations used in each time segment
\item
  changeProb: A numeric value representing the change probability for
  each of the bands used for change detection
\item
  *\_coefs: The regression coefficients for each of the bands in the
  input image collection
\item
  *\_rmse: The model root-mean-square error for each time segment and
  input band
\item
  *\_magnitude: For time segments with detected changes, this represents
  the normalized residuals during the change period
\end{itemize}

Notice that next to the band name and band type, there is also the
number of dimensions (i.e., 1 dimension, 2 dimensions). This is an
indication that we are dealing with an array image, which typically
requires a specific set of functions for proper manipulation, some of
which we will use in the next steps. We will start by looking at the
change bands, which are one of the key outputs of the CCDC algorithm. We
will select the band containing the information on the timing of break,
and find the number of breaks for a given time range. In the same
script, paste the code below:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Select time of break and change probability array images.  }
\KeywordTok{var}\NormalTok{ change }\OperatorTok{=}\NormalTok{ ccdResults}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}tBreak\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ changeProb }\OperatorTok{=}\NormalTok{ ccdResults}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}changeProb\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Set the time range we want to use and get as mask of  }
\CommentTok{// places that meet the condition.  }
\KeywordTok{var}\NormalTok{ start }\OperatorTok{=} \DecValTok{2000}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ end }\OperatorTok{=} \DecValTok{2021}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ mask }\OperatorTok{=}\NormalTok{ change}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(start)}\OperatorTok{.}\FunctionTok{and}\NormalTok{(change}\OperatorTok{.}\FunctionTok{lte}\NormalTok{(end))}\OperatorTok{.}\FunctionTok{and}\NormalTok{(changeProb}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(  }
\DecValTok{1}\NormalTok{))}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(changeProb}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}change prob\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Obtain the number of breaks for the time range.  }
\KeywordTok{var}\NormalTok{ numBreaks }\OperatorTok{=}\NormalTok{ mask}\OperatorTok{.}\FunctionTok{arrayReduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{sum}\NormalTok{()}\OperatorTok{,}\NormalTok{ [}\DecValTok{0}\NormalTok{])}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(numBreaks}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{5}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Number of breaks\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

With this code, we define the time range that we want to use, and then
we generate a mask that will indicate all the positions in the image
array with breaks detected in that range that also meet the condition of
having a change probability of 1, effectively removing some spurious
breaks. For each pixel, we can count the number of times that the mask
retrieved a valid result, indicating the number of breaks detected by
CCDC. In the loaded layer, places that appear brighter will show a
higher number of breaks, potentially indicating the conversion from
forest to agriculture, followed by multiple agricultural cycles. Keep in
mind that the detection of a break does not always imply a change of
land cover. Natural events, small-scale disturbances and seasonal
cycles, among others, can result in the detection of a break by CCDC.
Similarly, changes in the condition of the land cover in a pixel can
also be detected as breaks by CCDC, and some erroneous breaks can also
happen due to noisy time series or other factors.

For places with many changes, visualizing the first or last time when a
break was recorded can be helpful to understand the change dynamics
happening in the landscape. Paste the code below in the same script:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Obtain the first change in that time period.  }
\KeywordTok{var}\NormalTok{ dates }\OperatorTok{=}\NormalTok{ change}\OperatorTok{.}\FunctionTok{arrayMask}\NormalTok{(mask)}\OperatorTok{.}\FunctionTok{arrayPad}\NormalTok{([}\DecValTok{1}\NormalTok{])}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ firstChange }\OperatorTok{=}\NormalTok{ dates  }
   \OperatorTok{.}\FunctionTok{arraySlice}\NormalTok{(}\DecValTok{0}\OperatorTok{,} \DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([  }
\NormalTok{       [}\StringTok{\textquotesingle{}firstChange\textquotesingle{}}\NormalTok{]  }
\NormalTok{   ])  }
   \OperatorTok{.}\FunctionTok{selfMask}\NormalTok{()}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ timeVisParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{colorbrewer}\OperatorTok{.}\AttributeTok{YlOrRd}\NormalTok{[}\DecValTok{9}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:}\NormalTok{ start}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:}\NormalTok{ end  }
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(firstChange}\OperatorTok{,}\NormalTok{ timeVisParams}\OperatorTok{,} \StringTok{\textquotesingle{}First change\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Obtain the last change in that time period.  }
\KeywordTok{var}\NormalTok{ lastChange }\OperatorTok{=}\NormalTok{ dates  }
   \OperatorTok{.}\FunctionTok{arraySlice}\NormalTok{(}\DecValTok{0}\OperatorTok{,} \OperatorTok{{-}}\DecValTok{1}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([  }
\NormalTok{       [}\StringTok{\textquotesingle{}lastChange\textquotesingle{}}\NormalTok{]  }
\NormalTok{   ])  }
   \OperatorTok{.}\FunctionTok{selfMask}\NormalTok{()}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(lastChange}\OperatorTok{,}\NormalTok{ timeVisParams}\OperatorTok{,} \StringTok{\textquotesingle{}Last change\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Here we use arrayMask to keep only the change dates that meet our
condition, by using the mask we created previously. We use the function
arrayPad to fill or ``pad'' those pixels that did not experience any
change and therefore have no value in the tBreak band. Then we select
either the first or last values in the array, and we convert the image
from a one-dimensional array to a regular image, in order to apply a
visualization to it, using a custom palette. The results should look
like Fig. F4.7.2.

Finally, we can use the magnitude bands to visualize where and when the
largest changes as recorded by CCDC have occurred, during our selected
time period. We are going to use the magnitude of change in the SWIR1
band, masking it and padding it in the same way we did before. Paste
this code in your script:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Get masked magnitudes.  }
\KeywordTok{var}\NormalTok{ magnitudes }\OperatorTok{=}\NormalTok{ ccdResults  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SWIR1\_magnitude\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{arrayMask}\NormalTok{(mask)  }
   \OperatorTok{.}\FunctionTok{arrayPad}\NormalTok{([}\DecValTok{1}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Get index of max abs magnitude of change.  }
\KeywordTok{var}\NormalTok{ maxIndex }\OperatorTok{=}\NormalTok{ magnitudes  }
   \OperatorTok{.}\FunctionTok{abs}\NormalTok{()  }
   \OperatorTok{.}\FunctionTok{arrayArgmax}\NormalTok{()  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([  }
\NormalTok{       [}\StringTok{\textquotesingle{}index\textquotesingle{}}\NormalTok{]  }
\NormalTok{   ])}\OperatorTok{;}  
  
\CommentTok{// Select max magnitude and its timing  }
\KeywordTok{var}\NormalTok{ selectedMag }\OperatorTok{=}\NormalTok{ magnitudes}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{(maxIndex)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ selectedTbreak }\OperatorTok{=}\NormalTok{ dates}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{(maxIndex)}\OperatorTok{.}\FunctionTok{selfMask}\NormalTok{()}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ magVisParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{matplotlib}\OperatorTok{.}\AttributeTok{viridis}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.15}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.15}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(selectedMag}\OperatorTok{,}\NormalTok{ magVisParams}\OperatorTok{,} \StringTok{\textquotesingle{}Max mag\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(selectedTbreak}\OperatorTok{,}\NormalTok{ timeVisParams}\OperatorTok{,} \StringTok{\textquotesingle{}Time of max mag\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\includegraphics{./F4/image69.png}

\begin{figure}

{\centering \includegraphics{./F4/image1.png}

}

\caption{Fig. F4.7.2 First (top) and last (bottom) detected breaks for
the study area. Darker colors represent more recent dates, while
brighter colors represent older dates. The first change layer shows the
clear patterns of original agricultural expansion closer to the year
2000. The last change layer shows the more recently detected and noisy
breaks in the same areas. The thin areas in the center of the image have
only one time of change, corresponding to a single deforestation event.
Pixels with no detected breaks are masked and therefore show the basemap
underneath, set to show satellite imagery.}

\end{figure}

We first take the absolute value because the magnitudes can be positive
or negative, depending on the direction of the change and the band used.
For example, a positive value in the SWIR1 may show a forest loss event,
where surface reflectance goes from low to higher values. Brighter
values in Fig. 4.7.3 represent events of that type. Conversely, a
flooding event would have a negative value, due to the corresponding
drop in reflectance. Once we find the maximum absolute value, we find
its position on the array and then use that index to extract the
original magnitude value, as well as the time when that break occurred.

\begin{figure}

{\centering \includegraphics{./F4/image44.png}

}

\caption{Fig. F4.7.3 Maximum magnitude of change for the SWIR1 band for
the selected study period}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F47c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Question 2. Compare the ``first change'' and ``last change'' layers with
the layer showing the timing of the maximum magnitude of change. Use the
Inspector to check the values for specific pixels if necessary. What
does the timing of the layers tell you about the change processes
happening in the area?

Question 3. Looking at the ``max magnitude of change'' layer, find
places showing the largest and the smallest values. What type of changes
do you think are happening in each of those places?

\hypertarget{extracting-coefficients-manually}{%
\subsection{Extracting Coefficients
Manually}\label{extracting-coefficients-manually}}

In addition to the change information generated by the CCDC algorithm,
we can use the coefficients of the time segments for multiple purposes,
like land cover classification. Each time segment can be described as a
harmonic function with an intercept, slope, and three pairs of sine and
cosine terms that allow the time segments to represent seasonality
occurring at different temporal scales. These coefficients, as well as
the root-mean-square error (RMSE) obtained by comparing each predicted
and actual Landsat value, are produced when the CCDC algorithm is run.
The following example will show you how to retrieve the intercept
coefficient for a segment intersecting a specific date. In a new script,
paste the code below:

var palettes = require(`users/gena/packages:palettes');

var resultsPath =
`projects/gee-book/assets/F4-7/Rondonia\_example\_small';\\
var ccdResults = ee.Image(resultsPath);\\
Map.centerObject(ccdResults, 10);\\
print(ccdResults);

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Display segment start and end times.  }
\KeywordTok{var}\NormalTok{ start }\OperatorTok{=}\NormalTok{ ccdResults}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}tStart\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ end }\OperatorTok{=}\NormalTok{ ccdResults}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}tEnd\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(start}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1999}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{2001}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Segment start\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(end}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{2010}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{2020}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Segment end\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Check the Console and expand the bands section in the printed image
information. We will be using the tStart, tEnd, and SWIR1\_coefs bands,
which are array images containing the date when the time segments start,
date time segments end, and the coefficients for each of those segments
for the SWIR1 band. Run the code above and switch the map to Satellite
mode. Using the Inspector, click anywhere on the images, noticing the
number of dates printed and their values for multiple clicked pixels.
You will notice that for places with stable forest cover, there is
usually one value for tStart and one for tEnd. This means that for those
more stable places, only one time segment was fit by CCDC. On the other
hand, for places with visible transformation in the basemap, the number
of dates is usually two or three, meaning that the algorithm fitted two
or three time segments, respectively. To simplify the processing of the
data, we can select a single segment to extract its coefficients. Paste
the code below and re-run the script:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Find the segment that intersects a given date.  }
\KeywordTok{var}\NormalTok{ targetDate }\OperatorTok{=} \FloatTok{2005.5}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ selectSegment }\OperatorTok{=}\NormalTok{ start}\OperatorTok{.}\FunctionTok{lte}\NormalTok{(targetDate)}\OperatorTok{.}\FunctionTok{and}\NormalTok{(end}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(targetDate))}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(selectSegment}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Identified segment\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the code above, we set a time of interest, in this case the middle of
2005, and then we find the segments that meet the condition of starting
before and ending after that date. Using the Inspector again, click on
different locations and verify the outputs. The segment that meets the
condition will have a value of 1, and the other segments will have a
value of 0. We can use this information to select the coefficients for
that segment, using the code below:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Get all coefs in the SWIR1 band.  }
\KeywordTok{var}\NormalTok{ SWIR1Coefs }\OperatorTok{=}\NormalTok{ ccdResults}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SWIR1\_coefs\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(SWIR1Coefs}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1 coefs\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Select only those for the segment that we identified previously.  }
\KeywordTok{var}\NormalTok{ sliceStart }\OperatorTok{=}\NormalTok{ selectSegment}\OperatorTok{.}\FunctionTok{arrayArgmax}\NormalTok{()}\OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([  }
\NormalTok{   [}\StringTok{\textquotesingle{}index\textquotesingle{}}\NormalTok{]  }
\NormalTok{])}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ sliceEnd }\OperatorTok{=}\NormalTok{ sliceStart}\OperatorTok{.}\FunctionTok{add}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ selectedCoefs }\OperatorTok{=}\NormalTok{ SWIR1Coefs}\OperatorTok{.}\FunctionTok{arraySlice}\NormalTok{(}\DecValTok{0}\OperatorTok{,}\NormalTok{ sliceStart}\OperatorTok{,}\NormalTok{ sliceEnd)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(selectedCoefs}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Selected SWIR1 coefs\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the piece of code above, we first select the array image with the
coefficients for the SWIR1 band. Then, using the layer that we created
before, we find the position where the condition is true, and use that
to extract the coefficients only for that segment. Once again, you can
verify that using the Inspector tab.

Finally, what we have now is the full set of coefficients for the
segment that intersects the midpoint of 2005. The coefficients are in
the following order: intercept, slope, cosine 1, sine 1, cosine 2, sine
2, cosine 3, and sine 3. For this exercise we will extract the intercept
coefficient (Fig. 4.7.4), which is the first element in the array, using
the code below:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Retrieve only the intercept coefficient.  }
\KeywordTok{var}\NormalTok{ intercept }\OperatorTok{=}\NormalTok{ selectedCoefs}\OperatorTok{.}\FunctionTok{arraySlice}\NormalTok{(}\DecValTok{1}\OperatorTok{,} \DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{)}\OperatorTok{.}\FunctionTok{arrayProject}\NormalTok{([}\DecValTok{1}\NormalTok{])}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ intVisParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{matplotlib}\OperatorTok{.}\AttributeTok{viridis}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{6}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{6}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(intercept}\OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([  }
\NormalTok{   [}\StringTok{\textquotesingle{}INTP\textquotesingle{}}\NormalTok{]  }
\NormalTok{])}\OperatorTok{,}\NormalTok{ intVisParams}\OperatorTok{,} \StringTok{\textquotesingle{}INTP\_SWIR1\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image78.png}

}

\caption{Fig. F4.7.4 Values for the intercept coefficient of the
segments that start before and end after the midpoint of 2005}

\end{figure}

Since we run the CCDC algorithm on Landsat surface reflectance images,
intercept values should represent the average reflectance of a segment.
However, if you click on the image, you will see that the values are
outside of the 0--1 range. This is because the intercept is calculated
by the CCDC algorithm for the origin (e.g., time 0), and not for the
year we requested. In order to retrieve the adjusted intercept, as well
as other coefficients, we will use a different approach.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F47d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Section 5. Extracting Coefficients Using External Functions

The code we generated in the previous section allowed us to extract a
single coefficient for a single date. However, we typically want to
extract a set of multiple coefficients and bands that we can use as
inputs to other workflows, such as classification. To simplify that
process, we will use the same function library that we saw in Sect. 2.
In this section we will extract and visualize different coefficients for
a single date and produce an RGB image using the intercept coefficients
for multiple spectral bands for the same date. The first step involves
determining the date of interest and converting the CCDC results from
array images to regular multiband images for easier manipulation and
faster display. In a new script, copy the code below:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load the required libraries.  }
\KeywordTok{var}\NormalTok{ palettes }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:palettes\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ utils }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(   }\StringTok{\textquotesingle{}users/parevalo\_bu/gee{-}ccdc{-}tools:ccdcUtilities/api\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Load the results.  }
\KeywordTok{var}\NormalTok{ resultsPath }\OperatorTok{=}   \StringTok{\textquotesingle{}projects/gee{-}book/assets/F4{-}7/Rondonia\_example\_small\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ ccdResults }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(resultsPath)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(ccdResults}\OperatorTok{,} \DecValTok{10}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Convert a date into fractional years.  }
\KeywordTok{var}\NormalTok{ inputDate }\OperatorTok{=} \StringTok{\textquotesingle{}2005{-}09{-}25\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ dateParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{inputFormat}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
   \DataTypeTok{inputDate}\OperatorTok{:}\NormalTok{ inputDate}\OperatorTok{,}  
   \DataTypeTok{outputFormat}\OperatorTok{:} \DecValTok{1}  
\NormalTok{\}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ formattedDate }\OperatorTok{=}\NormalTok{ utils}\OperatorTok{.}\AttributeTok{Dates}\OperatorTok{.}\FunctionTok{convertDate}\NormalTok{(dateParams)}\OperatorTok{;}  
  
\CommentTok{// Band names originally used as inputs to the CCD algorithm.  }
\KeywordTok{var}\NormalTok{ BANDS }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}BLUE\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}GREEN\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}RED\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Names for the time segments to retrieve.  }
\KeywordTok{var}\NormalTok{ SEGS }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}S1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S7\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}S9\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}S10\textquotesingle{}}  
\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Transform CCD results into a multiband image.  }
\KeywordTok{var}\NormalTok{ ccdImage }\OperatorTok{=}\NormalTok{ utils}\OperatorTok{.}\AttributeTok{CCDC}\OperatorTok{.}\FunctionTok{buildCcdImage}\NormalTok{(ccdResults}\OperatorTok{,}\NormalTok{ SEGS}\OperatorTok{.}\AttributeTok{length}\OperatorTok{,}  
\NormalTok{   BANDS)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(ccdImage)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the code above we define the date of interest (2005-09-25) and
convert it to the date format in which we ran CCDC, which corresponds to
fractional years. After that, we specify the band that we used as inputs
for the CCDC algorithm. Finally, we specify the names we will assign to
the time segments, with the list length indicating the maximum number of
time segments to retrieve per pixel. This step is done because the
results generated by CCDC are stored as variable-length arrays. For
example, a pixel where there are no breaks detected will have one time
segment, but another pixel where a single break was detected may have
one or two segments, depending on when the break occurred. Requesting a
pre-defined maximum number of segments ensures that the structure of the
multi-band image is known, and greatly facilitates its manipulation and
display. Once we have set these variables, we call a function that
converts the result into an image with several bands representing the
combination of segments requested, input bands, and coefficients. You
can see the image structure in the Console.

Finally, to extract a subset of coefficients for the desired bands, we
can use a function in the imported library, called getMultiCoefs. This
function expects the following ordered parameters:

\begin{itemize}
\tightlist
\item
  The CCDC results in the multiband format we just generated in the step
  above.
\item
  The date for which we want to extract the coefficients, in the format
  in which the CCDC results were run (fractional years in our case).
\item
  List of the bands to retrieve (i.e., spectral bands).
\item
  List of coefficients to retrieve, defined as follows: INTP
  (intercept), SLP (slope), COS, SIN,COS32, SIN2, COS3, SIN3, and RMSE.
\item
  A Boolean flag of true or false, indicating whether we want the
  intercepts to be calculated for the input date, instead of being
  calculated at the origin. If true, SLP must be included in the list of
  coefficients to retrieve.
\item
  List of segment names, as used to create the multiband image in the
  prior step.
\item
  Behavior to apply if there is no time segment for the requested date:
  normal will retrieve a value only if the date intersects a segment;
  before or after will use the value of the segment immediately before
  or after the requested date, if no segment intersects the date
  directly.
\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define bands to select.  }
\KeywordTok{var}\NormalTok{ SELECT\_BANDS }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}RED\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}GREEN\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}BLUE\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Define coefficients to select.  }
\CommentTok{// This list contains all possible coefficients, and the RMSE  }
\KeywordTok{var}\NormalTok{ SELECT\_COEFS }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}INTP\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SLP\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}RMSE\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Obtain coefficients.  }
\KeywordTok{var}\NormalTok{ coefs }\OperatorTok{=}\NormalTok{ utils}\OperatorTok{.}\AttributeTok{CCDC}\OperatorTok{.}\FunctionTok{getMultiCoefs}\NormalTok{(  }
\NormalTok{   ccdImage}\OperatorTok{,}\NormalTok{ formattedDate}\OperatorTok{,}\NormalTok{ SELECT\_BANDS}\OperatorTok{,}\NormalTok{ SELECT\_COEFS}\OperatorTok{,} \KeywordTok{true}\OperatorTok{,}  
\NormalTok{   SEGS}\OperatorTok{,} \StringTok{\textquotesingle{}after\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(coefs)}\OperatorTok{;}  
  
\CommentTok{// Show a single coefficient.  }
\KeywordTok{var}\NormalTok{ slpVisParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{matplotlib}\OperatorTok{.}\AttributeTok{viridis}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.0005}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.005}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(coefs}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}RED\_SLP\textquotesingle{}}\NormalTok{)}\OperatorTok{,}\NormalTok{ slpVisParams}\OperatorTok{,}   \StringTok{\textquotesingle{}RED SLOPE 2005{-}09{-}25\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ rmseVisParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{matplotlib}\OperatorTok{.}\AttributeTok{viridis}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.1}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(coefs}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NIR\_RMSE\textquotesingle{}}\NormalTok{)}\OperatorTok{,}\NormalTok{ rmseVisParams}\OperatorTok{,}   \StringTok{\textquotesingle{}NIR RMSE 2005{-}09{-}25\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Show an RGB with three coefficients.  }
\KeywordTok{var}\NormalTok{ rgbVisParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}RED\_INTP\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}GREEN\_INTP\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}BLUE\_INTP\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.1}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(coefs}\OperatorTok{,}\NormalTok{ rgbVisParams}\OperatorTok{,} \StringTok{\textquotesingle{}RGB 2005{-}09{-}25\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The slope and RMSE images are shown in Fig. 4.7.5. For the slopes, high
positive values are bright, while large negative values are very dark.
Most of the remaining forest is stable and has a slope close to zero,
while areas that have experienced transformation and show agricultural
activity tend to have positive slopes in the RED band, appearing bright
in the image. Similarly, for the RMSE image, stable forests present more
predictable time series of surface reflectance that are captured more
faithfully by the time segments, and therefore present lower RMSE
values, appearing darker in the image. Agricultural areas present
noisier time series that are more challenging to model, and result in
higher RMSE values, appearing brighter.

\includegraphics{./F4/image18.png}

\begin{figure}

{\centering \includegraphics{./F4/image28.png}

}

\caption{Fig. F4.7.5 Image showing the slopes (top) and RMSE (bottom) of
the segments that intersect the requested date}

\end{figure}

Finally, the RGB image we created is shown in Fig. 4.7.6. The intercepts
are calculated for the middle point of the time segment intercepting the
date we requested, representing the average reflectance for the span of
the selected segment. In that sense, when shown together as an RGB
image, they are similar to a composite image for the selected date, with
the advantage of always being cloud-free.

\begin{figure}

{\centering \includegraphics{./F4/image76.png}

}

\caption{Fig. F4.7.6 RGB image created using the time segment intercepts
for the requested date}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F47e. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-14}{%
\subsection*{Conclusion}\label{conclusion-14}}
\addcontentsline{toc}{subsection}{Conclusion}

This chapter provided a guide for the interpretation of the results from
the CCDC algorithm for studying deforestation in the Amazon. Consider
the advantages of such an analysis compared to traditional approaches to
change detection, which are typically based on the comparison of two or
a few images collected over the same area. For example, with time-series
analysis, we can study trends and subtle processes such as vegetation
recovery or degradation, determine the timing of land-surface events,
and move away from retrospective analyses to monitoring in near-real
time. Through the use of all available clear observations, CCDC can
detect intra-annual breaks and capture seasonal patterns, although at
the expense of increased computational requirements and complexity,
unlike faster and easier to interpret methods based on annual
composites, such as LandTrendr (Chap. F4.5). We expect to see more
applications that make use of multiple change detection approaches (also
known as ``Ensemble'' approaches), and multisensor analyses in which
data from different satellites are fused (radar and optical, for
example) for higher data density.

\hypertarget{references-10}{%
\subsection*{References}\label{references-10}}
\addcontentsline{toc}{subsection}{References}

Arévalo P, Bullock EL, Woodcock CE, Olofsson P (2020) A suite of tools
for continuous land change monitoring in Google Earth Engine. Front Clim
2. https://doi.org/10.3389/fclim.2020.576740

Box GEP, Jenkins GM, Reinsel GC (1994) Time Series Analysis: Forecasting
and Control. Prentice Hall

Gorelick N, Hancher M, Dixon M, et al (2017) Google Earth Engine:
Planetary-scale geospatial analysis for everyone. Remote Sens Environ
202:18--27. https://doi.org/10.1016/j.rse.2017.06.031

Kennedy RE, Andréfouët S, Cohen WB, et al (2014) Bringing an ecological
view of change to Landsat-based remote sensing. Front Ecol Environ
12:339--346. https://doi.org/10.1890/130066

Woodcock CE, Loveland TR, Herold M, Bauer ME (2020) Transitioning from
change detection to monitoring with remote sensing: A paradigm shift.
Remote Sens Environ 238:111558.
https://doi.org/10.1016/j.rse.2019.111558

Zhu Z, Woodcock CE (2014) Continuous change detection and classification
of land cover using all available Landsat data. Remote Sens Environ
144:152--171. https://doi.org/10.1016/j.rse.2014.01.011

\hypertarget{data-fusion-merging-classification-streams}{%
\section{Data Fusion: Merging Classification
Streams}\label{data-fusion-merging-classification-streams}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-13}{%
\subsubsection*{Author}\label{author-13}}

Jeffrey A. Cardille, Rylan Boothman, Mary Villamor, Elijah Perez, Eidan
Willis, Flavie Pelletier

\hypertarget{overview-15}{%
\subsubsection*{Overview}\label{overview-15}}

As the ability to rapidly produce classifications of satellite images
grows, it will be increasingly important to have algorithms that can
sift through them to separate the signal from inevitable classification
noise. The purpose of this chapter is to explore how to update
classification time series by blending information from multiple
classifications made from a wide variety of data sources. In this lab,
we will explore how to update the classification time series of the
Roosevelt River found in Fortin et al.~(2020). That time series began
with the 1972 launch of Landsat 1, blending evidence from 10 sensors and
more than 140 images to show the evolution of the area until 2016. How
has it changed since 2016? What new tools and data streams might we tap
to understand the land surface through time?

\hypertarget{learning-outcomes-15}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-15}}

\begin{itemize}
\tightlist
\item
  Distinguishing between merging sensor data and merging classifications
  made from sensors.
\item
  Working with the Bayesian Updating of Land Cover (BULC) algorithm, in
  its basic form, to blend classifications made across multiple years
  and sensors.
\item
  Working with the BULC-D algorithm to highlight locations that changed.
\end{itemize}

\hypertarget{assumes-you-know-how-to-15}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-15}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks, classify images (Part F2).
\item
  \hspace{0pt}\hspace{0pt}Create a graph using ui.Chart (Chap. F1.3).
\item
  Obtain accuracy metrics from classifications (Chap. F2.2).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-12}{%
\subsection*{Introduction}\label{introduction-12}}

When working with multiple sensors, we are often presented with a
challenge: What to do with classification noise? It's almost impossible
to remove all noise from a classification. Given the information
contained in a stream of classifications, however, you should be able to
use the temporal context to distinguish noise from true changes in the
landscape.

The Bayesian Updating of Land Cover (BULC) algorithm (Cardille and
Fortin 2016) is designed to extract the signal from the noise in a
stream of classifications made from any number of data sources. BULC's
principal job is to estimate, at each time step, the likeliest state of
land use and land cover (LULC) in a study area given the accumulated
evidence to that point. It takes a stack of provisional classifications
as input; in keeping with the terminology of Bayesian statistics, these
are referred to as ``Events,'' because they provide new evidence to the
system. BULC then returns a stack of classifications as output that
represents the estimated LULC time series implied by the Events.

BULC estimates, at each time step, the most likely class from a set
given the evidence up to that point in time. This is done by employing
an accuracy assessment matrix like that seen in Chap. F2.2. At each time
step, the algorithm quantifies the agreement between two classifications
adjacent in time within a time series.

If the Events agree strongly, they are evidence of the true condition of
the landscape at that point in time. If two adjacent Events disagree,
the accuracy assessment matrix limits their power to change the class of
a pixel in the interpreted time series. As each new classification is
processed, BULC judges the credibility of a pixel's stated class and
keeps track of a set of estimates of the probability of each class for
each pixel. In this way, each pixel traces its own LULC history,
reflected through BULC's judgment of the confidence in each of the
classifications. The specific mechanics and formulas of BULC are
detailed in Cardille and Fortin (2016).

BULC's code is written in JavaScript, with modules that weigh evidence
for and against change in several ways, while recording parts of the
data-weighing process for you to inspect. In this lab, we will explore
BULC through its graphical user interface (GUI), which allows rapid
interaction with the algorithm's main functionality.

\hypertarget{imagery-and-classifications-of-the-roosevelt-river}{%
\subsection{Imagery and Classifications of the Roosevelt
River}\label{imagery-and-classifications-of-the-roosevelt-river}}

How has the Roosevelt River area changed in recent decades? One way to
view the area's recent history is to use Google Earth Timelapse, which
shows selected annual clear images of every part of Earth's terrestrial
surface since the 1980s. (You can find the site quickly with a web
search.) Enter ``Roosevelt River, Brazil'' in the search field. For
centuries, this area was very remote from agricultural development. It
was so little known to Westerners that when former US President Theodore
Roosevelt traversed it in the early 1900s there was widespread doubt
about whether his near-death experience there was exaggerated or even
entirely fictional (Millard 2006). After World War II, the region saw
increased agricultural development. Fortin et al.~(2020) traced four
decades of the history of this region with satellite imagery. Timelapse,
meanwhile, indicates that land cover conversion continued after 2016.
Can we track it using Earth Engine?

In this section, we will view the classification inputs to BULC, which
were made separately from this lab exercise by identifying training
points and classifying them using Earth Engine's regression tree
capability. As seen in Table 4.8.1, the classification inputs included
Sentinel-2 optical data, Landsat 7, Landsat 8, and the Advanced
Spaceborne Thermal Emission and Reflection Radiometer (ASTER) aboard
Terra. Though each classification was made with care, they each contain
noise, with each pixel likely to have been misclassified one or more
times. This could lead us to draw unrealistic conclusions if the
classifications themselves were considered as a time series. For
example, we would judge it highly unlikely that an area represented by a
pixel would really be agriculture one day and revert to intact forest
later in the month, only to be converted to agriculture again soon
after, and so on. With careful (though unavoidably imperfect)
classifications, we would expect that an area that had truly been
converted to agriculture would consistently be classified as
agriculture, while an area that remained as forest would be classified
as that class most of the time. BULC's logic is to detect that
persistence, extracting the true LULC change and stability from the
noisy signal of the time series of classifications.

Table F4.8.1 Images classified for updating Roosevelt River LULC with
BULC

Sensor

Date

Spatial resolution

Sentinel-2

2016: February 8

2017: July 7

2018: May 28

2019: June 17, June 17

2020: May 27, May 27

2021: May 27, July 11, August 15

10m

Landsat 7

2017: August 16

30m

Landsat 8

2021: July 18

30m

ASTER

2017: July 15

2018: August 19

2019: June 19

2020: August 8

15m--30m

As you have seen in earlier chapters, creating classifications can be
very involved and time consuming. To allow you to concentrate on BULC's
efforts to clean noise from an existing ImageCollection, we have created
the classifications already and stored them as an ImageCollection asset.
You can view the Event time series using the ui.Thumbnail function,
which creates an animation of the elements of the
collection\hspace{0pt}\hspace{0pt}\hspace{0pt}\hspace{0pt}. Paste the
code below into a new script to see those classifications drawn in
sequence in the Console.

var events = ee.ImageCollection(
`projects/gee-book/assets/F4-8/cleanEvents');\\
print(events, `List of Events');\\
print(`Number of events:', events.size());

print(ui.Thumbnail(events, \{\\
min: 0,\\
max: 3,\\
palette: {[}`black', `green', `blue', `yellow'{]},\\
framesPerSecond: 1,\\
dimensions: 1000\\
\}));

In the thumbnail sequence, the color palette shows Forest (class 1) as
green, Water (class 2) as blue, and Active Agriculture (class 3) as
yellow. Areas with no data in a particular Event are shown in black.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F48a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{basics-of-the-bulc-interface}{%
\subsection{Basics of the BULC
Interface}\label{basics-of-the-bulc-interface}}

To see if BULC can successfully sift through these Events, we will use
BULC's GUI (Fig. F4.8.1), which makes interacting with the functionality
straightforward. :::\{.callout-note\} Code Checkpoint F48b in the book's
repository contains information about accessing that interface.

\end{tcolorbox}

\begin{figure}

{\centering \includegraphics{./F4/image5.png}

}

\caption{Fig. F4.8.1 BULC interface}

\end{figure}

After you have run the script, BULC's interface requires that a few
parameters be set; these are specified using the left panel. Here, we
describe and populate each of the required parameters, which are shown
in red. As you proceed, the default red color will change to green when
a parameter receives a value.

\begin{itemize}
\tightlist
\item
  The interface permits new runs to be created using the Manual or
  Automated methods. The Automated setting allows information from a
  previous run to be used without manual entry. In this tutorial, we
  will enter each parameter individually using the interface, so you
  should set this item to Manual by clicking once on it.
\item
  Select type of image: The interface can accept pre-made Event inputs
  in one of three forms: (1) as a stored ImageCollection; (2) as a
  single multi-banded Image; and (3) as a stream of Dynamic World
  classifications. The classifications are processed in the order they
  are given, either within the ImageCollection or sequentially through
  the Image bands. For this run, select Image Collection from the
  dropdown menu, then enter the path to this collection, without
  enclosing it in quotes: projects/gee-book/assets/F4-8/cleanEvents
\item
  Remap: in some settings, you might want to remap the input value to
  combine classes. Leave this empty for now; an example of this is
  discussed later in the lab.
\item
  Number of Classes in Events and Number of Classes to Track: The
  algorithm requires the number of classes in each Event and the number
  of meaningful classes to track to be entered. Here, there are 3
  classes in each classification (Forest, Water, and Active Agriculture)
  and 3 classes being tracked. (In the BULC-U version of the algorithm
  (Lee et al.~2018, 2020), these numbers may be different when the
  Events are made using unsupervised classifications, which may contain
  many more classes than are being tracked in a given run.) Meaningful
  classes are assumed by BULC to begin with 1 rather than 0, while class
  values of 0 in Events are treated as no data. As seen in the thumbnail
  of Events, there are 3 classes; set both of these values to 3.
\item
  The Default Study Area is used by BULC to delimit the location to
  analyze. This value can be pulled from a specially sized Image or set
  automatically, using the extent of the inputs. Set this parameter to
  Event Geometry, which gets the value automatically from the Event
  collection.
\item
  The Base Land Cover Image defines the initial land cover condition to
  which BULC adds evidence from Events. Here, we are working to update
  the land cover map from the end of 2016, as estimated in Fortin et
  al.~(2020). The ending estimated classification from that study has
  been loaded as an asset and placed as the first image in the input
  ImageCollection. We will direct the BULC interface to use this first
  image in the collection as the base land cover image by selecting Top.
\item
  Overlay Approach: BULC can run in multiple modes, which affect the
  outcome of the classification updating. One option, Overlay, overlays
  each consecutive Event with the one prior in the sequence, following
  Fortin et al.~(2016). Another option, Custom, allows a user-defined
  constant array to be used. For this tutorial, we will choose D
  (Identity matrix), which uses the same transition table for every
  Event, regardless of how it overlays with the Event prior. That table
  gives a large conditional likelihood to the chance that classes agree
  strongly across the consecutive Event classifications that are used as
  inputs.
\end{itemize}

BULC makes relatively small demands on memory since its arithmetic uses
only multiplication, addition, and division, without the need for
complex function fitting. The specific memory use is tied to the overlay
method used. In particular, Event-by-Event comparisons (the Overlay
setting) are considerably more computationally expensive than
pre-defined transition tables (the Identity and Custom settings). The
maximum working Event depth is also slightly lowered when intermediate
probability values are returned for inspection. Our tests indicate that
with pre-defined truth tables and no intermediate probability values
returned, BULC can handle updating problems hundreds of Events deep
across an arbitrarily large area.

\begin{itemize}
\tightlist
\item
  Initialization Approach: If a BULC run of the full intended size ever
  surpassed the memory available, you would be able to break the
  processing into two or more parts using the following technique.
  First, create a slightly smaller run that can complete, and save the
  final probability image of that run. Because of the operation of
  Bayes' theorem, the ending probability multi-band image can be used as
  the prior probability to continue processing Events with the same
  answers as if it had been run all at once. For this small run, we will
  select F (First Run).
\item
  Levelers: BULC uses three levelers as part of its processing, as
  described in Fortin et al.~(2016). The Initialization Leveler creates
  the initial probability vector of the initial LULC image; the
  Transition Leveler dampens transitions at each time step, making BULC
  less reactive to new evidence; and the Posterior Leveler ensures that
  each class retains nonzero probability so that the Bayes formula can
  function properly throughout the run. For this run, set the parameters
  to 0.65, 0.3, and 0.6, respectively. This corresponds to a typical set
  of values that is appropriate when moderate-quality classifications
  are fed to BULC.
\item
  Colour Output Palette: We will use the same color palette as what was
  seen in the small script you used to draw the Events, with one
  exception. Because BULC will give a value for the estimated class for
  every pixel, there are no pixels in the study area with missing or
  masked data. To line up the colors with the attainable numbers, we
  will remove the color `black' from the specification. For this field,
  enter this list: {[}`green', `blue', `yellow'{]}. For all of the text
  inputs, make sure to click outside that field after entering text so
  that the input information is registered; the changing of the text
  color to green confirms that the information was received.
\end{itemize}

When you have finished setting the required parameters, the interface
will look like Fig. 4.8.2.

\begin{figure}

{\centering \includegraphics{./F4/image62.png}

}

\caption{Fig. 4.8.2 Initial settings for the key driving parameters of
BULC}

\end{figure}

Beneath the required parameters is a set of optional parameters that
affect which intermediate results are stored during a run for later
inspection. We are also given a choice of returning intermediate results
for closer inspection. At this stage, you can leave all optional
parameters out of the BULC call by leaving them blanked or unchecked.

After clicking the Apply Parameters button at the bottom of the left
panel, the classifications and parameters are sent to the BULC modules.
The Map will move to the study area, and after a few seconds, the
Console will hold new thumbnails. The uppermost thumbnail is a rapidly
changing view of the input classifications. Beneath that is a thumbnail
of the same area as interpreted by BULC. Beneath those is a Confidence
thumbnail, which is discussed in detail later in this lab.

The BULC interpretation of the landscape looks roughly like the Event
inputs, but it is different in two important ways. First, depending on
the leveler settings, it will usually have less noise than the Event
classifications. In the settings above, we used the Transition and
Posterior levelers to tell BULC to trust past accumulated evidence more
than a single new image. The second key difference between the BULC
result and the input classifications is that even when the inputs don't
cover the whole area at each time step, BULC provides an estimate in
every pixel at each time step. To create this continuous classification,
if a new classification does not have data for some part of the study
area (beyond the edge of a given image, for example), the last best
guess from the previous iteration is carried forward. Simply put, the
estimate in a given pixel is kept the same until new data arrives.

Meanwhile, below the Console, the rest of the interface changes when
BULC is run. The Map panel displays BULC's classification for the final
date: that is, after considering the evidence from each of the input
classifications. We can use the Satellite background to judge whether
BULC is accurately capturing the state of LULC. This can be done by
unselecting the drawn layers in the map layer set and selecting
Satellite from the choices in the upper-right part of the Map panel.
Earth Engine's background satellite images are often updated, so you
should see something like the right side of Fig. F4.8.3, though it may
differ slightly.

\includegraphics{./F4/image57.png}

\begin{figure}

{\centering \includegraphics{./F4/image65.png}

}

\caption{Fig. 4.8.3 BULC estimation of the state of LULC at the end of
2021 (left). Satellite backdrop for Earth Engine (right), which may
differ from what you see due to updates.}

\end{figure}

Question 1. When comparing the BULC classification for 2021 against the
current Earth Engine satellite view, what are the similarities and
differences? Note that in Earth Engine, the copyrighted year numbers at
the bottom of the screen may not coincide with the precise date of the
image shown.

In the rightmost panel below the Console, the interface offers you
multiple options for viewing the results. These include:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Movie. This uses the ui.Thumbnail API function to draw the BULC
  results rapidly in the viewer. This option offers you a control on the
  frame rate (in frames per second), and a checkbox affecting the
  drawing resolution. The high-resolution option uses the maximum
  resolution permitted given the function's constraints. A lower
  resolution setting constructs the thumbnail more quickly, but at a
  loss of detail.
\item
  Filmstrip. This produces an image like the Movie option, but allows
  you to move on request through each image.
\item
  Mosaic. This draws every BULC result in the panel. Depending on the
  size of the stack of classifications, this could become quite a large
  set of images.
\item
  Zoom. This draws the final BULC classification at multiple scales,
  with the finest-scale image matching that shown in the Map window.
\end{enumerate}

Question 2. Select the BULC option, then select the Movie tool to view
the result, and choose a drawing speed and resolution. When viewing the
full area, would you assess the additional LULC changes since 2016 as
being minor, moderate, or major compared to the changes that occurred
before 2016? Explain the reasoning for your assessment.

\hypertarget{detailed-lulc-inspection-with-bulc}{%
\subsection{Detailed LULC Inspection with
BULC}\label{detailed-lulc-inspection-with-bulc}}

BULC results can be viewed interactively, allowing you to view more
detailed estimations of the LULC around the study area. We will zoom
into a specific area where change did occur after 2016. To do that, turn
on the Satellite view and zoom in. Watching the scale bar in the lower
right of the Map panel, continue zooming until the scale bar says 5 km.
Then, enter ``-60.742, -9.844'' in the Earth Engine search tool, located
above the code. The text will be interpreted as a longitude/latitude
value and will offer you a nearby coordinate, indicated with a value for
the degrees West and the degrees South. Click that entry and Earth
Engine will move to that location, while keeping at the specified zoom
level. Let's compare the BULC result in this sector against the image
from Earth Engine's satellite view that is underneath it (Fig. 4.8.4).

\includegraphics{./F4/image51.png}

\begin{figure}

{\centering \includegraphics{./F4/image10.png}

}

\caption{Fig. 4.8.4 Comparison of the final classification of the
northern part of the study area to the satellite view}

\end{figure}

BULC captured the changes between 2016 and 2021 with a classification
series that suggests agricultural development (Fig. 4.8.4, left). Given
the appearance of BULC's 2021 classification, it suggests that the
satellite backdrop at the time of this writing (Fig. 4.8.4, right) came
from an earlier time period.

Now, in the Results panel, select BULC, then Movie. Set your desired
frame speed and resolution, then select Redraw Thumbnail. Then, zoom the
main Map even closer to some agriculture that appears to have been
established between 2016 and 2021. Redraw the thumbnail movie as needed
to find an interesting set of pixels.

With this finer-scale access to the results of BULC, you can select
individual pixels to inspect. Move the horizontal divider downward to
expose the Inspector tab and Console tab. Use the Inspector to click on
several pixels to learn their history as expressed in the inputted
Events and in BULC's interpretation of the noise and signal in the Event
series. In a chosen pixel, you might see output that looks like Fig.
4.8.5. It indicates a possible conversion in the Event time series after
a few classifications of the pixel as Forest. This decreases the
confidence that the pixel is still Forest (Fig. 4.8.5, lower panel), but
not enough for the Active Agriculture class (class 3) to become the
dominant probability. After the subsequent Event labels the pixel as
Forest, the confidence (lower panel) recovers slightly, but not to its
former level. The next Event classifies the pixel as Active Agriculture,
confidently, by interpreting that second Active Agriculture
classification, in a setting where change was already somewhat suspected
after the first non-Forest classification. BULC's label (middle panel)
changes to be Active Agriculture at that point in the sequence.
Subsequent Event classifications as Active Agriculture creates a growing
confidence that its proper label at the end of the sequence was indeed
Active Agriculture.

\includegraphics{./F4/image23.png}\includegraphics{./F4/image72.png}\includegraphics{./F4/image58.png}

Question 3. Run the code again with the same data, but adjust the three
levelers, then view the results presented in the Map window and the
Results panel. How do each of the three parameters affect the behavior
of BULC in its results? Use the thumbnail to assess your subjective
satisfaction with the results, and use the Inspector to view the BULC
behavior in individual pixels. Can you produce an optimal outcome for
this given set of input classifications?

\hypertarget{change-detection-with-bulc-d}{%
\subsection{Change Detection with
BULC-D}\label{change-detection-with-bulc-d}}

What if we wanted to identify areas of likely change or stability
without trying to identify the initial and final LULC class? BULC-D is
an algorithm that estimates, at each time step, the probability of
noteworthy change. The example below uses the Normalized Burn Ratio
(NBR) as a gauge: BULC-D assesses whether the ratio has meaningfully
increased, decreased, or remained the same. It is then the choice of the
analyst to decide how to treat these assessed probabilities of stability
and change.

BULC-D involves determining an expectation for an index across a
user-specified time period and then comparing new values against that
estimation. Using Bayesian logic, BULC-D then asks which of three
hypotheses is most likely, given evidence from the new values to date
from that index. The hypotheses are simple: Either the value has
decreased meaningfully, or it has increased meaningfully, or it has not
changed substantially compared to the previously established
expectation. The details of the workings of BULC-D are beyond the scope
of this exercise, but we provide it as a tool for exploration. BULC-D's
basic framework is the following:

\begin{itemize}
\tightlist
\item
  Establish: Fit a harmonic curve with a user-specified number of terms
  to a stream of values from an index, such as the Normalized Difference
  Vegetation Index (NDVI), NBR, etc.
\item
  Standardize: For each new image, quantify the deviation of the index's
  value from the expectation on that date.
\item
  Contextualize: Assess the magnitude of that deviation in one of
  several ordered bins.
\item
  Synthesize: Use the BULC framework to adjust the vector of change for
  the three possibilities: the value went down, the value stayed the
  same, the value went up.
\end{itemize}

It is worth noting that BULC-D does not label the change with a LULC
category; rather, it trains itself to distinguish likely LULC change
from expected variability. In this way, BULC-D can be thought of as a
``sieve'' through which you are able to identify locations of possible
change, isolated from likely background noise. In the BULC-D stage, the
likeliness of change is identified across the landscape; in a separate
second stage, the meaning of those changes and any changes to LULC
classes are identified. We will explore the workings of BULC-D using its
GUI.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F48c. The book's repository contains information about
accessing that interface.

\end{tcolorbox}

After you have run the script to initialize the interface, BULC-D's
interface requires a few parameters to be set. For this run of BULC-D,
we will set the parameters to the following:

\begin{itemize}
\tightlist
\item
  Expectation years: 2020
\item
  Target year: 2021
\item
  Sensors: Landsat and Sentinel
\item
  Index: NBR
\item
  Harmonic fit: Yes, 1 harmonic term.
\end{itemize}

Run BULC-D for this area. As a reminder, you should first zoom in enough
that the scale bar reads ``5 km'' or finer. Then, search for the
location ``-60.7624, -9.8542''. When you run BULC-D, a result like Fig.
F4.8.6 is shown for the layer of probabilities.

\includegraphics{./F4/image48.png}

Fig. 4.8.6 Result for BULC-D for the Roosevelt River area, depicting
estimated probability of change and stability for 2021

The BULC-D image (Fig. F4.8.6) shows each pixel as a continuous
three-value vector along a continuous range; the three values sum to 1.
For example, a vector with values of {[}0.85, 0.10, 0.05{]} would
represent an area estimated with high confidence according to BULC-D to
have experienced a sustained drop in NBR in the target period compared
to the values set by the expectation data. In that pixel, the
combination of three colors would produce a value that is richly red.
You can see Chap. F1.1 for more information on drawing bands of
information to the screen using the red-green-blue additive color model
in Earth Engine.

Each pixel experiences its own NBR history in both the expectation
period and the target year. Next, we will highlight the history of three
nearby areas: one, marked with a red balloon in your interface, that
BULC assessed as having experienced a persistent drop in NBR; a second
in green assessed to not have changed, and a third in blue assessed to
have witnessed a persistent NBR increase.

Figure F4.8.7 shows the NBR history for the red balloon in the southern
part of the study area in Fig. F4.8.4. If you click on that pixel or one
like it, you can see that, whereas the values were quite stable
throughout the growing season for the years used to create the pixel's
expectation, they were persistently lower in the target year. This is
flagged as a likely meaningful drop in the NBR by BULC-D, for
consideration by the analyst.

\includegraphics{./F4/image79.png}

Fig. 4.8.7 NBR history for a pixel with an apparent drop in NBR in the
target year (below) as compared to the expectation years (above). Pixel
is colored a shade of red in Fig. 4.8.6.

Figure F4.8.8 shows the NBR history for the blue balloon in the southern
part of the study area in Fig. F4.8.4. For that pixel, while the values
were quite stable throughout the growing season for the years used to
create the pixel's expectation, they were persistently higher in the
target year.

Question 4. Experiment with turning off one of the satellite sensor data
sources used to create the expectation collection. For example, do you
get the same results if the Sentinel-2 data stream is not used, or is
the outcome different. You might make screen captures of the results to
compare with Fig. 4.8.4. How strongly does each satellite stream affect
the outcome of the estimate? Do differences in the resulting estimate
vary across the study area?

\begin{figure}

{\centering \includegraphics{./F4/image52.png}

}

\caption{Fig. 4.8.8 NBR history for a pixel with an apparent increase in
NBR in the target year (below) as compared to the expectation years
(above). Pixel is colored a shade of blue in Fig. 4.8.6.}

\end{figure}

Figure F4.8.8 also shows that, for that pixel, the fit of values for the
years used to build the expectation showed a sine wave (shown in blue),
but with a fit that was not very strong. When data for the target year
was assembled (Fig. F4.8.8, bottom), the values were persistently above
expectation throughout the growing season. Note that this pixel was
identified as being different in the target year as compared to earlier
years, which does not rule out the possibility that the LULC of the area
was changed (for example, from Forest to Agriculture) during the years
used to build the expectation collection. BULC-D is intended to be run
steadily over a long period of time, with the changes marked as they
occur, after which point the expectation would be recalculated.

\begin{figure}

{\centering \includegraphics{./F4/image56.png}

}

\caption{Fig. 4.8.9 NBR history for a pixel with no apparent increase or
decrease in NBR in the target year (below) as compared to the
expectation years (above). Pixel is colored a shade of green in Fig.
4.8.6.}

\end{figure}

Fig. F4.8.9 shows the NBR history for the green balloon in the southern
part of the study area in Fig. F4.8.4. For that pixel, the values in the
expectation collection formed a sine wave, and the values in the target
collection deviated only slightly from the expectation during the target
year.

\hypertarget{change-detection-with-bulc-and-dynamic-world}{%
\subsection{Change Detection with BULC and Dynamic
World}\label{change-detection-with-bulc-and-dynamic-world}}

Recent advances in neural networks have made it easier to develop
consistent models of LULC characteristics using satellite data. The
Dynamic World project (Brown et al.~2022) applies a neural network,
trained on a very large number of images, to each new Sentinel-2 image
soon after it arrives. The result is a near-real-time classification
interpreting the LULC of Earth's surface, kept continually up to date
with new imagery.

What to do with the inevitable inconsistencies in a pixel's stated LULC
class through time? For a given pixel on a given image, its assigned
class label is chosen by the Dynamic World algorithm as the maximum
class probability given the band values on that day. Individual class
probabilities are given as part of the dataset and could be used to
better interpret a pixel's condition and perhaps its history. Future
work with BULC will involve incorporating these probabilities into
BULC's probability-based structure. For this tutorial, we will explore
the consistency of the assigned labels in this same Roosevelt River area
as a way to illustrate BULC's potential for minimizing noise in this
vast and growing dataset.

\hypertarget{using-bulc-to-explore-and-refine-dynamic-world-classifications}{%
\subsubsection{5.1. Using BULC To Explore and Refine Dynamic World
Classifications}\label{using-bulc-to-explore-and-refine-dynamic-world-classifications}}

Code Checkpoint A48d. The book's repository contains a script to use to
begin this section. You will need to load the linked script and run it
to begin.

After running the linked script, the BULC interface will initialize.
Select Dynamic World from the dropdown menu where you earlier selected
Image Collection. When you do, the interface opens several new fields to
complete. BULC will need to know where you are interested in working
with Dynamic World, since it could be anywhere on Earth. To specify the
location, the interface field expects a nested list of lists of lists,
which is modeled after the structure used inside the constructor
ee.Geometry.Polygon. (When using drawing tools or specifying study areas
using coordinates, you may have noticed this structure.) Enter the
following nested list in the text field near the Dynamic World option,
without enclosing it in quotes:

{[}{[}{[}-61.155, -10.559{]}, {[}-60.285, -10.559{]}, {[}-60.285,
-9.436{]}, {[}-61.155, -9.436{]}{]}{]}

Next, BULC will need to know which years of Dynamic World you are
interested in. For this exercise, select 2021. Then, BULC will ask for
the Julian days of the year that you are interested in. For this
exercise, enter 150 for the start day and 300 for the end day. Because
you selected Dynamic World for analysis in BULC, the interface defaults
to offering the number 9 for the number of classes in Events and for the
number of classes to track. This number represents the full set of
classes in the Dynamic World classification scheme. You can leave other
required settings shown in green with their default values. For the
Color Output Palette, enter the following palette without quotes. This
will render results in the Dynamic World default colors.

{[}`419BDF', `397D49', `88B053', `7A87C6', `E49635', `DFC35A', `C4281B',
`A59B8F', `B39FE1'{]}

When you have finished, select Apply Parameters at the bottom of the
input panel. When it runs, BULC subsets the Dynamic World dataset to
clip out according to the dates and location, identifying images from
more than 40 distinct dates. The area covers two of the tiles in which
Dynamic World classifications are partitioned to be served, so BULC
receives more than 90 classifications. When BULC finishes its run, the
Map panel will look like Fig. F4.8.10, BULC's estimate of the final
state of the landscape at the end of the classification sequence.

\includegraphics{./F4/image37.png}\includegraphics{./F4/image41.png}

Let's explore the suite of information returned by BULC about this time
period in Dynamic World. Enter ``Muiraquitã'' in the search bar and view
the results around that area to be able to see the changing LULC
classifications within farm fields. Then, begin to inspect the results
by viewing a Movie of the Events, with a data frame rate of 6 frames per
second. Because the study area spans multiple Dynamic World tiles, you
will find that many Event frames are black, meaning that there was no
data in your sector on that particular image. Because of this, and also
perhaps because of the very aggressive cloud masking built into Dynamic
World, viewing Events (which, as a reminder, are the individual
classified images directly from Dynamic World) can be a very challenging
way to look for change and stability. BULC's goal is to sift through
those classifications to produce a time series that reflects, according
to its estimation, the most likely LULC value at each time step. View
the Movie of the BULC results and ask yourself whether each class is
equally well replicated across the set of classifications. A still from
midway through the Movie sequence of the BULC results can be seen in
Fig. F4.8.11.

\begin{figure}

{\centering \includegraphics{./F4/image13.png}

}

\caption{Fig. F4.8.11 Still frame (right image) from the animation of
BULC's adjusted estimate of LULC through time near Muiraquitã}

\end{figure}

As BULC uses the classification inputs to estimate the state of the LULC
at each time step, it also tracks its confidence in those estimates.
This is shown in several ways in the interface.

\begin{itemize}
\tightlist
\item
  You can view a Movie of BULC's confidence through time as it reacts to
  the consistency or variability of the class identified in each pixel
  by Dynamic World. View that movie now over this area to see the
  evolution of BULC's confidence through time of the class of each
  pixel. A still frame from this movie can be seen in Fig. F4.8.12. The
  frame and animation indicate that BULC's confidence is lowest in
  pixels where the estimate flips between similar categories, such as
  Grass and Shrub \& Scrub. It also is low at the edges of land covers,
  even where the covers (such as Forest and Water) are easy to discern
  from each other.\\
\item
  You can inspect the final confidence estimate from BULC, which is
  shown as a grayscale image in the set of Map layers in the left lower
  panel. That single layer synthesizes how, across many Dynamic World
  classifications, the confidence in certain LULC classes and locations
  is ultimately more stable than in others. For example, generally
  speaking, the Forest class is classified consistently across this
  assemblage of Dynamic World images. Agriculture fields are less
  consistently classified as a single class, as evidenced by their
  relatively low confidence.
\item
  Another way of viewing BULC's confidence is through the Inspector tab.
  You can click on individual pixels to view their values in the Event
  time series and in the BULC time series, and see BULC's corresponding
  confidence value changing through time in response to the relative
  stability of each pixel's classification.
\item
  Another way to view BULC's confidence estimation is as a hillshade
  enhancement of the final BULC classification. If you select the
  Probability Hillshade in the set of Map layers, it shows the final
  BULC classification as a textured surface, in which you can see where
  lower-confidence pixels are classified.
\end{itemize}

\begin{figure}

{\centering \includegraphics{./F4/image40.png}

}

\caption{Fig. F4.8.12 Still frame from the animation of changing
confidence through time, near Muiraquitã.}

\end{figure}

\hypertarget{using-bulc-to-visualize-uncertainty-of-dynamic-world-in-simplified-categories}{%
\subsubsection{5.2. Using BULC To Visualize Uncertainty of Dynamic World
in Simplified
Categories}\label{using-bulc-to-visualize-uncertainty-of-dynamic-world-in-simplified-categories}}

In the previous section, you may have noticed that there are two main
types of uncertainty in BULC's assessment of long-term classification
confidence. One type is due to spatial uncertainty at the edge of two
relatively distinct phenomena, like the River/Forest boundary visible in
Fig. F4.8.12. These are shown in dark tones in the confidence images,
and emphasized in the Probability Hillshade. The other type of
uncertainty is due to some cause of labeling uncertainty, due either (1)
to the similarity of the classes, or (2) to persistent difficulty in
distinguishing two distinct classes that are meaningfully different but
spectrally similar. An example of uncertainty due to similar labels is
distinguishing flooded and non-flooded wetlands in classifications that
contain both those categories. An example of difficulty distinguishing
distinct but spectrally similar classes might be distinguishing a
parking lot from a body of water.

BULC allows you to remap the classifications it is given as input,
compressing categories as a way to minimize uncertainty due to
similarity among classes. In the setting of Dynamic World in this study
area, we notice that several classes are functionally similar for the
purposes of detecting new deforestation: Farm fields and pastures are
variously labeled on any given Dynamic World classification as Grass,
Flooded Vegetation, Crops, Shrub \& Scrub, Built, or Bare Ground. What
if we wanted to combine these categories to be similar to the
distinctions of the classified Events from this lab's Sect. 1? The
classes in that section were Forest, Water, and Active Agriculture. To
remap the Dynamic World classification, continue with the same run as in
Sect. 5.1. Near where you specified the location for clipping Dynamic
World, there are two fields for remapping. Select the Remap checkbox and
in the ``from'' field, enter (without quotes):

0,1,2,3,4,5,6,7,8

In the ``to'' field, enter (without quotes):

1,0,2,2,2,2,2,2,0

This directs BULC to create a three-class remap of each Dynamic World
image. Next, in the area of the interface where you specify the palette,
enter the same palette used earlier:

{[}`green', `blue', `yellow'{]}

Before continuing, think for a moment about how many classes you have
now. From BULC's perspective, the Dynamic World events will have 3
classes and you will be tracking 3 classes. Set both the Number of
Classes in Events and Number of Classes to Track to 3. Then click Apply
Parameters to send this new run to BULC.

The confidence image shown in the main Map panel is instructive (Fig.
4.8.13). Using data from 2020, 2021, and 2022, It indicates that much of
the uncertainty among the original Dynamic World classifications was in
distinguishing labels within agricultural fields. When that uncertainty
is removed by combining classes, the BULC result indicates that a
substantial part of the remaining uncertainty is at the edges of
distinct covers. For example, in the south-central and southern part of
the frame, much of the uncertainty among classifications in the original
Dynamic World classifications was due to distinction among the highly
similar, easily confused classes. Much of what remained (right) after
remapping (right) formed outlines of the river and the edges between
farmland and forest: a graphic depiction of the ``spatial uncertainty''
discussed earlier. Yet not all of the uncertainty was spatial; the
thicker, darker areas of uncertainty even after remapping (right, at the
extreme eastern edge for example) indicates a more fundamental
disagreement in the classifications. In those pixels, even when the
Agriculture-like classes were compressed, there was still considerable
uncertainty (likely between Forest and Active Agriculture) in the true
state of these areas. These might be of further interest: were they
places newly deforested in 2020-2022? Were they abandoned fields
regrowing? Were they degraded at some point? The mapping of uncertainty
may hold promise for a better understanding of uncertainty as it is
encountered in real classifications, thanks to Dynamic World.

\includegraphics{./F4/image47.png}\includegraphics{./F4/image89.png}

Given the tools and approaches presented in this lab, you should now be
able to import your own classifications for BULC (Sects. 1--3), detect
changes in sets of raw imagery (Sect. 4), or use Dynamic World's
pre-created classifications (Sect. 5). The following exercises explore
this potential.

\hypertarget{conclusion-15}{%
\subsection*{Conclusion}\label{conclusion-15}}
\addcontentsline{toc}{subsection}{Conclusion}

In this lab, you have viewed several related but distinct ways to use
Bayesian statistics to identify locations of LULC change in complex
landscapes. While they are standalone algorithms, they are each intended
to provide a perspective either on the likelihood of change (BULC-D) or
of extracting signal from noisy classifications (BULC). You can consider
using them especially when you have pixels that, despite your best
efforts, periodically flip back and forth between similar but different
classes. BULC can help ignore noise, and BULC-D can help reveal whether
this year's signal has precedent in past years.

To learn more about the BULC algorithm, you can view this interactive
probability illustration tool by a link found in script F48s1 -
Supplemental in the book's repository. In the future, after you have
learned how to use the logic of BULC, you might prefer to work with the
JavaScript code version. To do that, you can find a tutorial at the
website of the authors.

\hypertarget{references-11}{%
\subsection*{References}\label{references-11}}
\addcontentsline{toc}{subsection}{References}

Brown CF, Brumby SP, Guzder-Williams B, et al (2022) Dynamic World, Near
real-time global 10 m land use land cover mapping. Sci Data 9:1--17.
https://doi.org/10.1038/s41597-022-01307-4

Cardille JA, Fortin JA (2016) Bayesian updating of land-cover estimates
in a data-rich environment. Remote Sens Environ 186:234--249.
https://doi.org/10.1016/j.rse.2016.08.021

Fortin JA, Cardille JA, Perez E (2020) Multi-sensor detection of
forest-cover change across 45\,years in Mato Grosso, Brazil. Remote Sens
Environ 238. https://doi.org/10.1016/j.rse.2019.111266

Lee J, Cardille JA, Coe MT (2020) Agricultural expansion in Mato Grosso
from 1986-2000: A Bayesian time series approach to tracking past land
cover change. Remote Sens 12. https://doi.org/10.3390/rs12040688

Lee J, Cardille JA, Coe MT (2018) BULC-U: Sharpening resolution and
improving accuracy of land-use/land-cover classifications in Google
Earth Engine. Remote Sens 10. https://doi.org/10.3390/rs10091455

Millard C (2006) The River of Doubt: Theodore Roosevelt's Darkest
Journey. Anchor

\hypertarget{exploring-lagged-effects-in-time-series}{%
\section{Exploring Lagged Effects in Time
Series}\label{exploring-lagged-effects-in-time-series}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-14}{%
\subsubsection*{Author}\label{author-14}}

Andréa Puzzi Nicolau, Karen Dyson, David Saah, Nicholas Clinton

\hypertarget{overview-16}{%
\subsubsection*{Overview}\label{overview-16}}

In this chapter, we will introduce lagged effects to build on previous
work in modeling time-series data. Time-lagged effects occur when an
event at one point in time impacts dependent variables at a later point
in time. You will be introduced to concepts of autocovariance and
autocorrelation, cross-covariance and cross-correlation, and
auto-regressive models. At the end of this chapter, you will be able to
examine how variables relate to one another across time, and to fit time
series models that take into account lagged events.

\hypertarget{learning-outcomes-16}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-16}}

\begin{itemize}
\tightlist
\item
  Using the ee.Join function to create time-lagged collections.
\item
  Calculating autocovariance and autocorrelation.
\item
  Calculating cross-covariance and cross-correlation.
\item
  Fitting auto-regressive models.
\end{itemize}

\hypertarget{assumes-you-know-how-to-16}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-16}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks, classify images (Part F2).
\item
  \hspace{0pt}\hspace{0pt}Create a graph using ui.Chart (Chap. F1.3).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Mask cloud, cloud shadow, snow/ice, and other undesired pixels (Chap.
  F4.3).
\item
  Fit linear and nonlinear functions with regression in an
  ImageCollection time series (Chap. F4.6).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-13}{%
\subsection*{Introduction}\label{introduction-13}}

While fitting functions to time series allows you to account for
seasonality in your models, sometimes the impact of a seasonal event
does not impact your dependent variable until the next month, the next
year, or even multiple years later. For example, coconuts take 18--24
months to develop from flower to harvestable size. Heavy rains during
the flower development stage can severely reduce the number of coconuts
that can be harvested months later, with significant negative economic
repercussions. These patterns---where events in one time period impact
our variable of interest in later time periods---are important to be
able to include in our models.

In this chapter, we introduce lagged effects into our previous
discussions on interpreting time-series data (Chaps. F4.6 and F4.7).
Being able to integrate lagged effects into our time-series models
allows us to address many important questions. For example, streamflow
can be accurately modeled by taking into account previous streamflow,
rainfall, and soil moisture; this improved understanding helps predict
and mitigate the impacts of drought and flood events made more likely by
climate change (Sazib et al.~2020). As another example, time-series lag
analysis was able to determine that decreased rainfall was associated
with increases in livestock disease outbreaks one year later in India
(Karthikeyan et al.~2021).

\hypertarget{autocovariance-and-autocorrelation}{%
\subsection{Autocovariance and
Autocorrelation}\label{autocovariance-and-autocorrelation}}

Before we dive into autocovariance and autocorrelation, let's set up an
area of interest and dataset that we can use to illustrate these
concepts. We will work with a detrended time series (as seen in Chap.
F4.6) based on the USGS Landsat 8 Level 2, Collection 2, Tier 1 image
collection. Copy and paste the code below to filter the Landsat 8
collection to a point of interest over California and specific dates,
and apply the pre-processing function---to mask clouds (as seen in Chap.
F4.3) and to scale and add variables of interest (as seen in Chap.
F4.6).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define function to mask clouds, scale, and add variables  }
\CommentTok{// (NDVI, time and a constant) to Landsat 8 imagery.  }
\KeywordTok{function} \FunctionTok{maskScaleAndAddVariable}\NormalTok{(image) \{   }\CommentTok{// Bit 0 {-} Fill   // Bit 1 {-} Dilated Cloud   // Bit 2 {-} Cirrus   // Bit 3 {-} Cloud   // Bit 4 {-} Cloud Shadow   var qaMask = image.select(\textquotesingle{}QA\_PIXEL\textquotesingle{}).bitwiseAnd(parseInt(\textquotesingle{}11111\textquotesingle{},       2)).eq(0);   var saturationMask = image.select(\textquotesingle{}QA\_RADSAT\textquotesingle{}).eq(0);   // Apply the scaling factors to the appropriate bands.   var opticalBands = image.select(\textquotesingle{}SR\_B.\textquotesingle{}).multiply(0.0000275).add({-}       0.2);   var thermalBands = image.select(\textquotesingle{}ST\_B.*\textquotesingle{}).multiply(0.00341802)  }
       \OperatorTok{.}\FunctionTok{add}\NormalTok{(}\FloatTok{149.0}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Replace the original bands with the scaled ones and apply the masks.   var img = image.addBands(opticalBands, null, true)  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(thermalBands}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(qaMask)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(saturationMask)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ imgScaled }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(img}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Now we start to add variables of interest.   // Compute time in fractional years since the epoch.   var date = ee.Date(image.get(\textquotesingle{}system:time\_start\textquotesingle{}));   var years = date.difference(ee.Date(\textquotesingle{}1970{-}01{-}01\textquotesingle{}), \textquotesingle{}year\textquotesingle{});   var timeRadians = ee.Image(years.multiply(2 * Math.PI));  }
   \CommentTok{// Return the image with the added bands.  }
   \ControlFlowTok{return}\NormalTok{ imgScaled  }
       \CommentTok{// Add an NDVI band.  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(imgScaled}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\NormalTok{])  }
           \OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{))       }\CommentTok{// Add a time band.       .addBands(timeRadians.rename(\textquotesingle{}t\textquotesingle{}))  }
       \OperatorTok{.}\FunctionTok{float}\NormalTok{()       }\CommentTok{// Add a constant band.       .addBands(ee.Image.constant(1));  }
\NormalTok{\}  }
  
\CommentTok{// Import region of interest. Area over California.  }
\KeywordTok{var}\NormalTok{ roi }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{([  }
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{119.44617458417066}\OperatorTok{,}\FloatTok{35.92639730653253}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{119.07675930096754}\OperatorTok{,}\FloatTok{35.92639730653253}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{119.07675930096754}\OperatorTok{,}\FloatTok{36.201704711823844}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{119.44617458417066}\OperatorTok{,}\FloatTok{36.201704711823844}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{119.44617458417066}\OperatorTok{,}\FloatTok{35.92639730653253}\NormalTok{]  }
\NormalTok{])}\OperatorTok{;}  
  
  
\CommentTok{// Import the USGS Landsat 8 Level 2, Collection 2, Tier 1 collection,  }
\CommentTok{// filter, mask clouds, scale, and add variables.  }
\KeywordTok{var}\NormalTok{ landsat8sr }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(roi)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2013{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2018{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(maskScaleAndAddVariable)}\OperatorTok{;}  
  
\CommentTok{// Set map center.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(roi}\OperatorTok{,} \DecValTok{10}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, copy and paste the code below to estimate the linear trend using
the linearRegression reducer, and remove that linear trend from the time
series.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// List of the independent variable names.  }
\KeywordTok{var}\NormalTok{ independents }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{List}\NormalTok{([}\StringTok{\textquotesingle{}constant\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}t\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Name of the dependent variable.  }
\KeywordTok{var}\NormalTok{ dependent }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{String}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Compute a linear trend.  This will have two bands: \textquotesingle{}residuals\textquotesingle{} and  }
\CommentTok{// a 2x1 band called coefficients (columns are for dependent variables).  }
\KeywordTok{var}\NormalTok{ trend }\OperatorTok{=}\NormalTok{ landsat8sr}\OperatorTok{.}\FunctionTok{select}\NormalTok{(independents}\OperatorTok{.}\FunctionTok{add}\NormalTok{(dependent))  }
   \OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{linearRegression}\NormalTok{(independents}\OperatorTok{.}\FunctionTok{length}\NormalTok{()}\OperatorTok{,} \DecValTok{1}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Flatten the coefficients into a 2{-}band image  }
\KeywordTok{var}\NormalTok{ coefficients }\OperatorTok{=}\NormalTok{ trend}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}coefficients\textquotesingle{}}\NormalTok{)   }\CommentTok{// Get rid of extra dimensions and convert back to a regular image   .arrayProject([0])  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([independents])}\OperatorTok{;}  
  
\CommentTok{// Compute a detrended series.  }
\KeywordTok{var}\NormalTok{ detrended }\OperatorTok{=}\NormalTok{ landsat8sr}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{   }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(dependent)  }
       \OperatorTok{.}\FunctionTok{subtract}\NormalTok{(image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(independents)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(  }
\NormalTok{               coefficients)  }
           \OperatorTok{.}\FunctionTok{reduce}\NormalTok{(}\StringTok{\textquotesingle{}sum\textquotesingle{}}\NormalTok{))  }
       \OperatorTok{.}\FunctionTok{rename}\NormalTok{(dependent)  }
       \OperatorTok{.}\FunctionTok{copyProperties}\NormalTok{(image}\OperatorTok{,}\NormalTok{ [}\StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now let's turn to autocovariance and autocorrelation. The autocovariance
of a time series refers to the dependence of values in the time series
at time t with values at time h = t − lag. The autocorrelation is the
correlation between elements of a dataset at one time and elements of
the same dataset at a different time. The autocorrelation is the
autocovariance normalized by the standard deviations of the covariates.
Specifically, we assume our time series is stationary, and define the
autocovariance and autocorrelation following Shumway and Stoffer (2019).
Comparing values at time t to previous values is useful not only for
computing autocovariance, but also for a variety of other time series
analyses as you'll see shortly.

To combine image data with previous values in Earth Engine, the first
step is to join the previous values to the current values. To do that,
we will use a ee.Join function to create what we'll call a lagged
collection. Copy and paste the code below to define a function that
creates a lagged collection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Function that creates a lagged collection.  }
\KeywordTok{var}\NormalTok{ lag }\OperatorTok{=} \KeywordTok{function}\NormalTok{(leftCollection}\OperatorTok{,}\NormalTok{ rightCollection}\OperatorTok{,}\NormalTok{ lagDays) \{   }\KeywordTok{var}\NormalTok{ filter }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{and}\NormalTok{(       ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{maxDifference}\NormalTok{(\{  }
           \DataTypeTok{difference}\OperatorTok{:} \DecValTok{1000} \OperatorTok{*} \DecValTok{60} \OperatorTok{*} \DecValTok{60} \OperatorTok{*} \DecValTok{24} \OperatorTok{*}\NormalTok{ lagDays}\OperatorTok{,}  
           \DataTypeTok{leftField}\OperatorTok{:} \StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\OperatorTok{,}  
           \DataTypeTok{rightField}\OperatorTok{:} \StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\NormalTok{       \})}\OperatorTok{,}\NormalTok{       ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{greaterThan}\NormalTok{(\{  }
           \DataTypeTok{leftField}\OperatorTok{:} \StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\OperatorTok{,}  
           \DataTypeTok{rightField}\OperatorTok{:} \StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\NormalTok{       \}))}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Join}\OperatorTok{.}\FunctionTok{saveAll}\NormalTok{(\{  }
       \DataTypeTok{matchesKey}\OperatorTok{:} \StringTok{\textquotesingle{}images\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{measureKey}\OperatorTok{:} \StringTok{\textquotesingle{}delta\_t\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{ordering}\OperatorTok{:} \StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{ascending}\OperatorTok{:} \KeywordTok{false}\OperatorTok{,} \CommentTok{// Sort reverse chronologically   \}).apply(\{  }
       \DataTypeTok{primary}\OperatorTok{:}\NormalTok{ leftCollection}\OperatorTok{,}  
       \DataTypeTok{secondary}\OperatorTok{:}\NormalTok{ rightCollection}\OperatorTok{,}  
       \DataTypeTok{condition}\OperatorTok{:}\NormalTok{ filter  }
\NormalTok{   \})}\OperatorTok{;}  
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

This function joins a collection to itself, using a filter that gets all
the images before each image's date that are within a specified time
difference (in days) of each image. That list of previous images within
the lag time is stored in a property of the image called images, sorted
reverse chronologically. For example, to create a lagged collection from
the detrended Landsat imagery, copy and paste:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a lagged collection of the detrended imagery.  }
\KeywordTok{var}\NormalTok{ lagged17 }\OperatorTok{=} \FunctionTok{lag}\NormalTok{(detrended}\OperatorTok{,}\NormalTok{ detrended}\OperatorTok{,} \DecValTok{17}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Why 17 days? Recall that the temporal cadence of Landsat is 16 days.
Specifying 17 days in the join gets one previous image, but no more.

Now, we will compute the autocovariance using a reducer that expects a
set of one-dimensional arrays as input. So pixel values corresponding to
time t need to be stacked with pixel values at time t − lag as multiple
bands in the same image. Copy and paste the code below to define a
function to do so, and apply it to merge the bands from the lagged
collection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Function to stack bands.  }
\KeywordTok{var}\NormalTok{ merge }\OperatorTok{=} \KeywordTok{function}\NormalTok{(image) \{   }\CommentTok{// Function to be passed to iterate.   var merger = function(current, previous) \{       return ee.Image(previous).addBands(current);  }
\NormalTok{   \}}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{ImageCollection}\OperatorTok{.}\FunctionTok{fromImages}\NormalTok{(image}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}images\textquotesingle{}}\NormalTok{))  }
       \OperatorTok{.}\FunctionTok{iterate}\NormalTok{(merger}\OperatorTok{,}\NormalTok{ image)}\OperatorTok{;}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Apply merge function to the lagged collection.  }
\KeywordTok{var}\NormalTok{ merged17 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(lagged17}\OperatorTok{.}\FunctionTok{map}\NormalTok{(merge))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now the bands from time t and h are all in the same image. Note that the
band name of a pixel at time h, ph, was the same as time t, pt (band
name is ``NDVI'' in this case). During the merging process, it gets a
'\_1' appended to it (e.g.~NDVI\_1).

You can print the image collection to check the band names of one of the
images. Copy and paste the code below to map a function to convert the
merged bands to arrays with bands pt and ph, and then reduce it with the
covariance reducer. We use a parallelScale factor of 8 in the reduce
function to avoid the computation to run out of memory (this is not
always needed). Note that the output of the covariance reducer is an
array image, in which each pixel stores a 2x2 variance-covariance array.
The off-diagonal elements are covariance, which you can map directly
using the arrayGet function.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Function to compute covariance.  }
\KeywordTok{var}\NormalTok{ covariance }\OperatorTok{=} \KeywordTok{function}\NormalTok{(mergedCollection}\OperatorTok{,}\NormalTok{ band}\OperatorTok{,}\NormalTok{ lagBand) \{   }\ControlFlowTok{return}\NormalTok{ mergedCollection}\OperatorTok{.}\FunctionTok{select}\NormalTok{([band}\OperatorTok{,}\NormalTok{ lagBand])}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(  }
\NormalTok{       image) \{       }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{toArray}\NormalTok{()}\OperatorTok{;}  
\NormalTok{   \})}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{covariance}\NormalTok{()}\OperatorTok{,} \DecValTok{8}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Concatenate the suffix to the NDVI band.  }
\KeywordTok{var}\NormalTok{ lagBand }\OperatorTok{=}\NormalTok{ dependent}\OperatorTok{.}\FunctionTok{cat}\NormalTok{(}\StringTok{\textquotesingle{}\_1\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Compute covariance.  }
\KeywordTok{var}\NormalTok{ covariance17 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\FunctionTok{covariance}\NormalTok{(merged17}\OperatorTok{,}\NormalTok{ dependent}\OperatorTok{,}\NormalTok{ lagBand))  }
   \OperatorTok{.}\FunctionTok{clip}\NormalTok{(roi)}\OperatorTok{;}  
  
\CommentTok{// The output of the covariance reducer is an array image,  }
\CommentTok{// in which each pixel stores a 2x2 variance{-}covariance array.  }
\CommentTok{// The off diagonal elements are covariance, which you can map  }
\CommentTok{// directly using:  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(covariance17}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{([}\DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{])}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.02}\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}covariance (lag = 17 days)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Inspect the pixel values of the resulting covariance image (Fig.
F4.9.1). The covariance is positive when the greater values of one
variable (at time t) mainly correspond to the greater values of the
other variable (at time h), and the same holds for the lesser values,
therefore, the values tend to show similar behavior. In the opposite
case, when the greater values of a variable correspond to the lesser
values of the other variable, the covariance is negative.

\begin{figure}

{\centering \includegraphics{./F4/image2.png}

}

\caption{Fig. F4.9.1 Autocovariance image}

\end{figure}

The diagonal elements of the variance-covariance array are variances.
Copy and paste the code below to define and map a function to compute
correlation (Fig. F4.9.2) from the variance-covariance array.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define the correlation function.  }
\KeywordTok{var}\NormalTok{ correlation }\OperatorTok{=} \KeywordTok{function}\NormalTok{(vcArrayImage) \{   }\KeywordTok{var}\NormalTok{ covariance }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(vcArrayImage)}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{([}\DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{])}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ sd0 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(vcArrayImage)}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{([}\DecValTok{0}\OperatorTok{,} \DecValTok{0}\NormalTok{])}\OperatorTok{.}\FunctionTok{sqrt}\NormalTok{()}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ sd1 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(vcArrayImage)}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{([}\DecValTok{1}\OperatorTok{,} \DecValTok{1}\NormalTok{])}\OperatorTok{.}\FunctionTok{sqrt}\NormalTok{()}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ covariance}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(sd0)}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(sd1)}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(       }\StringTok{\textquotesingle{}correlation\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Apply the correlation function.  }
\KeywordTok{var}\NormalTok{ correlation17 }\OperatorTok{=} \FunctionTok{correlation}\NormalTok{(covariance17)}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(roi)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(correlation17}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}correlation (lag = 17 days)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image33.png}

}

\caption{Fig. F4.9.2 Autocorrelation image}

\end{figure}

Higher positive values indicate higher correlation between the elements
of the dataset, and lower negative values indicate the opposite.

It's worth noting that you can do this for longer lags as well. Of
course, that images list will fill up with all the images that are
within lag of t. Those other images are also useful---for example, in
fitting autoregressive models as described later.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F49a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{cross-covariance-and-cross-correlation}{%
\subsection{Cross-Covariance and
Cross-Correlation}\label{cross-covariance-and-cross-correlation}}

Cross-covariance is analogous to autocovariance, except instead of
measuring the correspondence between a variable and itself at a lag, it
measures the correspondence between a variable and a covariate at a lag.
Specifically, we will define the cross-covariance and cross-correlation
according to Shumway and Stoffer (2019).

You already have all the code needed to compute cross-covariance and
cross-correlation. But you do need a time series of another variable.
Suppose we postulate that NDVI is related in some way to the
precipitation before the NDVI was observed. To estimate the strength of
this relationship in every pixel, copy and paste the code below to the
existing script to load precipitation, join, merge, and reduce as
previously:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Precipitation (covariate)  }
\KeywordTok{var}\NormalTok{ chirps }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}UCSB{-}CHG/CHIRPS/PENTAD\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Join the t{-}l (l=1 pentad) precipitation images to the Landsat.  }
\KeywordTok{var}\NormalTok{ lag1PrecipNDVI }\OperatorTok{=} \FunctionTok{lag}\NormalTok{(landsat8sr}\OperatorTok{,}\NormalTok{ chirps}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add the precipitation images as bands.  }
\KeywordTok{var}\NormalTok{ merged1PrecipNDVI }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(lag1PrecipNDVI}\OperatorTok{.}\FunctionTok{map}\NormalTok{(merge))}\OperatorTok{;}  
  
\CommentTok{// Compute and display cross{-}covariance.  }
\KeywordTok{var}\NormalTok{ cov1PrecipNDVI }\OperatorTok{=} \FunctionTok{covariance}\NormalTok{(merged1PrecipNDVI}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}precipitation\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(roi)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(cov1PrecipNDVI}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{([}\DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{])}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}   \StringTok{\textquotesingle{}NDVI {-} PRECIP cov (lag = 5)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Compute and display cross{-}correlation.  }
\KeywordTok{var}\NormalTok{ corr1PrecipNDVI }\OperatorTok{=} \FunctionTok{correlation}\NormalTok{(cov1PrecipNDVI)}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(roi)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(corr1PrecipNDVI}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.5}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.5}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI {-} PRECIP corr (lag = 5)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

What do you observe from this result? Looking at the cross-correlation
image (Fig. F4.9.3), do you observe high values where you would expect
high NDVI values (vegetated areas)? One possible drawback of this
computation is that it's only based on five days of precipitation,
whichever five days came right before the NDVI image.

\begin{figure}

{\centering \includegraphics{./F4/image46.png}

}

\caption{Fig. F4.9.3 Cross-correlation image of NDVI and precipitation
with a five-day lag.}

\end{figure}

Perhaps precipitation in the month before the observed NDVI is relevant?
Copy and paste the code below to test the 30-day lag idea.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Join the precipitation images from the previous month.  }
\KeywordTok{var}\NormalTok{ lag30PrecipNDVI }\OperatorTok{=} \FunctionTok{lag}\NormalTok{(landsat8sr}\OperatorTok{,}\NormalTok{ chirps}\OperatorTok{,} \DecValTok{30}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ sum30PrecipNDVI }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(lag30PrecipNDVI}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(  }
\NormalTok{   image) \{   }\KeywordTok{var}\NormalTok{ laggedImages }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{ImageCollection}\OperatorTok{.}\FunctionTok{fromImages}\NormalTok{(image  }
       \OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}images\textquotesingle{}}\NormalTok{))}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(image)}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(laggedImages}\OperatorTok{.}\FunctionTok{sum}\NormalTok{()  }
       \OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}sum\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
\NormalTok{\}))}\OperatorTok{;}  
  
\CommentTok{// Compute covariance.  }
\KeywordTok{var}\NormalTok{ cov30PrecipNDVI }\OperatorTok{=} \FunctionTok{covariance}\NormalTok{(sum30PrecipNDVI}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}sum\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(  }
\NormalTok{   roi)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(cov1PrecipNDVI}\OperatorTok{.}\FunctionTok{arrayGet}\NormalTok{([}\DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{])}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}   \StringTok{\textquotesingle{}NDVI {-} sum cov (lag = 30)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Correlation.  }
\KeywordTok{var}\NormalTok{ corr30PrecipNDVI }\OperatorTok{=} \FunctionTok{correlation}\NormalTok{(cov30PrecipNDVI)}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(roi)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(corr30PrecipNDVI}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.5}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.5}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI {-} sum corr (lag = 30)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Observe that the only change is to the merge method. Instead of merging
the bands of the NDVI image and the covariate (precipitation), the
entire list of precipitation is summed and added as a band (eliminating
the need for iterate).

Which changes do you notice between the cross-correlation images---5
days lag vs.~30 days lag (Fig. F4.9.4)?. You can use the Inspector tool
to assess if the correlation increased or not at vegetated areas.

\begin{figure}

{\centering \includegraphics{./F4/image34.png}

}

\caption{Fig. F4.9.4 Cross-correlation image of NDVI and precipitation
with a 30-day lag.}

\end{figure}

As long as there is sufficient temporal overlap between the time series,
these techniques could be extended to longer lags and longer time
series.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F49b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{auto-regressive-models}{%
\subsection{Auto-Regressive Models}\label{auto-regressive-models}}

The discussion of autocovariance preceded this section in order to
introduce the concept of lag. Now that you have a way to get previous
values of a variable, it's worth considering auto-regressive models.
Suppose that pixel values at time t depend in some way on previous pixel
values---auto-regressive models are time series models that use
observations from previous time steps as input to a regression equation
to predict the value at the next time step. If you have observed
significant, non-zero autocorrelations in a time series, this is a good
assumption. Specifically, you may postulate a linear model such as the
following, where pt is a pixel at time t, and et is a random error
(Chap. F4.6):

pt = β0 + β1pt-1 + β2pt-2 + et (F4.9.1)

To fit this model, you need a lagged collection as created previously,
except with a longer lag (e.g., lag = 34 days). The next steps are to
merge the bands, then reduce with the linear regression reducer.

Copy and paste the line below to the existing script to create a lagged
collection, where the images list stores the two previous images:

var lagged34 = ee.ImageCollection(lag(landsat8sr, landsat8sr, 34));

Copy and paste the code below to merge the bands of the lagged
collection such that each image has bands at time t and bands at times t
- 1,\ldots, t − lag. Note that it's necessary to filter out any images
that don't have two previous temporal neighbors.

var merged34 = lagged34.map(merge).map(function(image) \{ return
image.set(`n', ee.List(image.get(`images'))\\
.length());\\
\}).filter(ee.Filter.gt(`n', 1));

Now, copy and paste the code below to fit the regression model using the
linearRegression reducer.

var arIndependents = ee.List({[}`constant', `NDVI\_1', `NDVI\_2'{]});

var ar2 = merged34\\
.select(arIndependents.add(dependent))\\
.reduce(ee.Reducer.linearRegression(arIndependents.length(), 1));

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Turn the array image into a multi{-}band image of coefficients.  }
\KeywordTok{var}\NormalTok{ arCoefficients }\OperatorTok{=}\NormalTok{ ar2}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}coefficients\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{arrayProject}\NormalTok{([}\DecValTok{0}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{arrayFlatten}\NormalTok{([arIndependents])}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We can compute the fitted values using the expression function in Earth
Engine. Because this model is a function of previous pixel values, which
may be masked, if any of the inputs to equation F4.9.1 are masked, the
output of the equation will also be masked. That's why you should use an
expression here, unlike the previous linear models of time. Copy and
paste the code below to compute the fitted values.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Compute fitted values.  }
\KeywordTok{var}\NormalTok{ fittedAR }\OperatorTok{=}\NormalTok{ merged34}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{   }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{addBands}\NormalTok{(  }
\NormalTok{       image}\OperatorTok{.}\FunctionTok{expression}\NormalTok{(           }\StringTok{\textquotesingle{}beta0 + beta1 * p1 + beta2 * p2\textquotesingle{}}\OperatorTok{,}\NormalTok{ \{  }
               \DataTypeTok{p1}\OperatorTok{:}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\_1\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
               \DataTypeTok{p2}\OperatorTok{:}\NormalTok{ image}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\_2\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
               \DataTypeTok{beta0}\OperatorTok{:}\NormalTok{ arCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}constant\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
               \DataTypeTok{beta1}\OperatorTok{:}\NormalTok{ arCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\_1\textquotesingle{}}\NormalTok{)}\OperatorTok{,}  
               \DataTypeTok{beta2}\OperatorTok{:}\NormalTok{ arCoefficients}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}NDVI\_2\textquotesingle{}}\NormalTok{)  }
\NormalTok{           \})}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}fitted\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Finally, copy and paste the code below to plot the results (Fig.
F4.9.5). We will use a specific point defined as pt.~Note the missing
values that result from masked data. If you run into computation errors,
try commenting the Map.addLayer calls from previous sections to save
memory.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create an Earth Engine point object to print the time series chart.  }
\KeywordTok{var}\NormalTok{ pt }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{119.0955}\OperatorTok{,} \FloatTok{35.9909}\NormalTok{])}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}\OperatorTok{.}\FunctionTok{series}\NormalTok{(  }
\NormalTok{       fittedAR}\OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}fitted\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NDVI\textquotesingle{}}\NormalTok{])}\OperatorTok{,}\NormalTok{ pt}\OperatorTok{,}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}  
   \OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,} \DecValTok{30}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{setSeriesNames}\NormalTok{([}\StringTok{\textquotesingle{}NDVI\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}fitted\textquotesingle{}}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{  }
       \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}AR(2) model: original and fitted values\textquotesingle{}}\OperatorTok{,}  
       \DataTypeTok{lineWidth}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
       \DataTypeTok{pointSize}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
\NormalTok{   \}))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F4/image19.png}

}

\caption{Fig. F4.9.5 Observed NDVI and fitted values at selected point}

\end{figure}

At this stage, note that the missing data has become a real problem. Any
data point for which at least one of the previous points is masked or
missing is also masked.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F49c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

It may be possible to avoid this problem by substituting the output from
equation F4.9.1 (the modeled value) for the missing or masked data.
Unfortunately, the code to make that happen is not straightforward. You
can check a solution in the following Code Checkpoint:

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F49d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-16}{%
\subsection*{Conclusion}\label{conclusion-16}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, we learned how to use autocovariance and
autocorrelation to explore the relationship between elements of a time
series at multiple time steps. We also explored how to use
cross-covariance and cross-correlation to examine the relationship
between elements of two time series at different points in time.
Finally, we used auto-regressive models to regress the elements of a
time series with elements of the same time series at a different point
in time. With these skills, you can now examine how events in one time
period impact your variable of interest in later time periods. While we
have introduced the linear approach to lagged effects, these ideas can
be expanded to more complex models.

\hypertarget{references-12}{%
\subsection*{References}\label{references-12}}
\addcontentsline{toc}{subsection}{References}

Karthikeyan R, Rupner RN, Koti SR, et al (2021) Spatio-temporal and time
series analysis of bluetongue outbreaks with environmental factors
extracted from Google Earth Engine (GEE) in Andhra Pradesh, India.
Transbound Emerg Dis 68:3631--3642. https://doi.org/10.1111/tbed.13972

Sazib N, Bolten J, Mladenova I (2020) Exploring spatiotemporal relations
between soil moisture, precipitation, and streamflow for a large set of
watersheds using Google Earth Engine. Water (Switzerland) 12:1371.
https://doi.org/10.3390/w12051371

Shumway RH, Stoffer DS (2019) Time Series: A Data Analysis Approach
Using R. Chapman and Hall/CRC

\hypertarget{vectors-and-tables}{%
\chapter{Vectors and Tables}\label{vectors-and-tables}}

In addition to raster data processing, Earth Engine supports a rich set
of vector processing tools. This Part introduces you to the vector
framework in Earth Engine, shows you how to create and to import your
vector data, and how to combine vector and raster data for analyses.

\hypertarget{exploring-vectors}{%
\section{Exploring Vectors}\label{exploring-vectors}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-15}{%
\subsubsection*{Author}\label{author-15}}

AJ Purdy, Ellen Brock, David Saah

\hypertarget{overview-17}{%
\subsubsection*{Overview}\label{overview-17}}

In this chapter, you will learn about features and feature collections
and how to use them in conjunction with images and image collections in
Earth Engine. Maps are useful for understanding spatial patterns, but
scientists often need to extract statistics to answer a question. For
example, you may make a false-color composite showing which areas of San
Francisco are more ``green''---i.e., have more healthy vegetation---than
others, but you will likely not be able to directly determine which
block in a neighborhood is the most green. This tutorial will
demonstrate how to do just that by utilizing vectors.

As described in Chap. F4.0, an important way to summarize and simplify
data in Earth Engine is through the use of reducers. Reducers operating
across space were used in Chap. F3.0, for example, to enable image
regression between bands. More generally, chapters in Part F3 and Part
F4 used reducers mostly to summarize the values across bands or images
on a pixel-by-pixel basis. What if you wanted to summarize information
within the confines of given spatial elements- for example, within a set
of polygons? In this chapter, we will illustrate and explore Earth
Engine's method for doing that, which is through a reduceRegions call.

\hypertarget{learning-outcomes-17}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-17}}

\begin{itemize}
\tightlist
\item
  Uploading and working with a shapefile as an asset to use in Earth
  Engine.
\item
  Creating a new feature using the geometry tools.
\item
  Importing and filtering a feature collection in Earth Engine.
\item
  Using a feature to clip and reduce image values within a geometry.
\item
  Use reduceRegions to summarize an image in irregular neighborhoods.
\item
  Exporting calculated data to tables with Tasks.
\end{itemize}

\hypertarget{assumes-you-know-how-to-17}{%
\subsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-17}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Calculate and interpret vegetation indices (Chap. F2.0).
\item
  Use drawing tools to create points, lines, and polygons (Chap. F2.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-14}{%
\subsection*{Introduction}\label{introduction-14}}

In the world of geographic information systems (GIS), data is typically
thought of in one of two basic data structures: raster and vector. In
previous chapters, we have principally been focused on raster
data---data using the remote sensing vocabulary of pixels, spatial
resolution, images, and image collections. Working within the vector
framework is also a crucial skill to master. If you don't know much
about GIS, you can find any number of online explainers of the
distinctions between these data types, their strengths and limitations,
and analyses using both data types. Being able to move fluidly between a
raster conception and a vector conception of the world is powerful, and
is facilitated with specialized functions and approaches in Earth
Engine.

For our purposes, you can think of vector data as information
represented as points (e.g., locations of sample sites), lines (e.g.,
railroad tracks), or polygons (e.g., the boundary of a national park or
a neighborhood). Line data and polygon data are built up from points:
for example, the latitude and longitude of the sample sites, the points
along the curve of the railroad tracks, and the corners of the park that
form its boundary. These points each have a highly specific location on
Earth's surface, and the vector data formed from them can be used for
calculations with respect to other layers. As will be seen in this
chapter, for example, a polygon can be used to identify which pixels in
an image are contained within its borders. Point-based data have already
been used in earlier chapters for filtering image collections by
location (see Part F1), and can also be used to extract values from an
image at a point or a set of points (see Chap. F5.2). Lines possess the
dimension of length and have similar capabilities for filtering image
collections and accessing their values along a transect. In addition to
using polygons to summarize values within a boundary, they can be used
for other, similar purposes---for example, to clip an image.

As you have seen, raster features in Earth Engine are stored as an Image
or as part of an ImageCollection. Using a similar conceptual model,
vector data in Earth Engine is stored as a Feature or as part of a
FeatureCollection. Features and feature collections provide useful data
to filter images and image collections by their location, clip images to
a boundary, or statistically summarize the pixel values within a region.

In the following example, you will use features and feature collections
to identify which city block near the University of San Francisco (USF)
campus is the most green.

\hypertarget{using-geometry-tools-to-create-features-in-earth-engine}{%
\subsection{Using Geometry Tools to Create Features in Earth
Engine}\label{using-geometry-tools-to-create-features-in-earth-engine}}

To demonstrate how geometry tools in Earth Engine work, let's start by
creating a point, and two polygons to represent different elements on
the USF campus.

Click on the geometry tools in the top left of the Map pane and create a
point feature. Place a new point where USF is located (see Fig. F5.0.1).

\begin{figure}

{\centering \includegraphics{./F5/image54.png}

}

\caption{Fig. F5.0.1 Location of the USF campus in San Francisco,
California. Your first point should be in this vicinity. The red arrow
points to the geometry tools.}

\end{figure}

Use Google Maps to search for ``Harney Science Center'' or ``Lo Schiavo
Center for Science.'' Hover your mouse over the Geometry Imports to find
the +new layer menu item and add a new layer to delineate the boundary
of a building on campus.

Next, create another new layer to represent the entire campus as a
polygon.

After you create these layers, rename the geometry imports at the top of
your script. Name the layers usf\_point, usf\_building, and usf\_campus.
These names are used within the script shown in Fig. F5.0.2.

\begin{figure}

{\centering \includegraphics{./F5/image10.png}

}

\caption{Fig. F5.0.2 Rename the default variable names for each layer in
the Imports section of the code at the top of your script}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F50a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{loading-existing-features-and-feature-collections-in-earth-engine}{%
\subsection{Loading Existing Features and Feature Collections in Earth
Engine}\label{loading-existing-features-and-feature-collections-in-earth-engine}}

If you wish to have the exact same geometry imports in this chapter for
the rest of this exercise, begin this section using the code at the Code
Checkpoint above.

Next, you will load a city block dataset to determine the amount of
vegetation on blocks near USF. The code below imports an existing
feature dataset in Earth Engine. The Topologically Integrated Geographic
Encoding and Referencing (TIGER) boundaries are census-designated
boundaries that are a useful resource when comparing socioeconomic and
diversity metrics with environmental datasets in the United States.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the Census Tiger Boundaries from GEE.  }
\KeywordTok{var}\NormalTok{ tiger }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}TIGER/2010/Blocks\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add the new feature collection to the map, but do not display.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(tiger}\OperatorTok{,}\NormalTok{ \{   }\StringTok{\textquotesingle{}color\textquotesingle{}}\OperatorTok{:} \StringTok{\textquotesingle{}black\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Tiger\textquotesingle{}}\OperatorTok{,} \KeywordTok{false}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

You should now have the geometry for USF's campus and a layer added to
your map that is not visualized for census blocks across the United
States. Next, we will use neighborhood data to spatially filter the
TIGER feature collection for blocks near USF's campus.

\hypertarget{importing-features-into-earth-engine}{%
\subsection{Importing Features into Earth
Engine}\label{importing-features-into-earth-engine}}

There are many image collections loaded in Earth Engine, and they can
cover a very large area that you might want to study. Borders can be
quite intricate (for example, a detailed coastline), and fortunately
there is no need for you to digitize the intricate boundary of a large
geographic area. Instead, we will show how to find a spatial dataset
online, download the data, and load this into Earth Engine as an asset
for use.

\hypertarget{find-a-spatial-dataset-of-san-francisco-neighborhoods}{%
\subsubsection{Find a Spatial Dataset of San Francisco
Neighborhoods}\label{find-a-spatial-dataset-of-san-francisco-neighborhoods}}

Use your internet searching skills to locate the ``Analysis
Neighborhoods'' dataset covering San Francisco. This data might be
located in a number of places, including DataSF, the City of San
Francisco's public-facing data repository.

\begin{figure}

{\centering \includegraphics{./F5/image27.png}

}

\caption{Fig. F5.0.3 DataSF website neighborhood shapefile to download}

\end{figure}

After you find the Analysis Neighborhoods layer, click Export and select
Shapefile (Fig. F5.0.3). Keep track of where you save the zipped file,
as we will load this into Earth Engine. Shapefiles contain vector-based
data---points, lines, polygons---and include a number of files, such as
the location information, attribute information, and others.

Extract the folder to your computer. When you open the folder, you will
see that there are actually many files. The extensions (.shp, .dbf,
.shx, .prj) all provide a different piece of information to display
vector-based data. The .shp file provides data on the geometry. The .dbf
file provides data about the attributes. The .shx file is an index file.
Lastly, the .prj file describes the map projection of the coordinate
information for the shapefile. You will need to load all four files to
create a new feature asset in Earth Engine.

\hypertarget{upload-sf-neighborhoods-file-as-an-asset}{%
\subsubsection{Upload SF Neighborhoods File as an
Asset}\label{upload-sf-neighborhoods-file-as-an-asset}}

Navigate to the Assets tab (near Scripts). Select New \textgreater{}
Table Upload \textgreater{} Shape files (Fig. F5.0.4).

\begin{figure}

{\centering \includegraphics{./F5/image52.png}

}

\caption{Fig. F5.0.4 Import an asset as a zipped folder}

\end{figure}

\hypertarget{select-files-and-name-asset}{%
\subsubsection{Select Files and Name
Asset}\label{select-files-and-name-asset}}

Click the Select button and then use the file navigator to select the
component files of the shapefile structure (i.e., .shp, .dbf, .shx, and
.prj) (Fig. F5.0.5). Assign an Asset Name so you can recognize this
asset.

\begin{figure}

{\centering \includegraphics{./F5/image43.png}

}

\caption{Fig. F5.0.5 Select the four files extracted from the zipped
folder. Make sure each file has the same name and that there are no
spaces in the file names of the component files of the shapefile
structure.}

\end{figure}

Uploading the asset may take a few minutes. The status of the upload is
presented under the Tasks tab. After your asset has been successfully
loaded, click on the asset in the Assets folder and find the collection
ID. Copy this text and use it to import the file into your Earth Engine
analysis.

Assign the asset to the table (collection) ID using the script below.
Note that you will need to replace `path/to/your/asset/assetname' with
the actual path copied in the previous step.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Assign the feature collection to the variable sfNeighborhoods.  }
\KeywordTok{var}\NormalTok{ sfNeighborhoods }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(   }\StringTok{\textquotesingle{}path/to/your/asset/assetname\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Print the size of the feature collection.  }
\CommentTok{// (Answers the question how many features?)  }
\FunctionTok{print}\NormalTok{(sfNeighborhoods}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfNeighborhoods}\OperatorTok{,}\NormalTok{ \{   }\StringTok{\textquotesingle{}color\textquotesingle{}}\OperatorTok{:} \StringTok{\textquotesingle{}blue\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}sfNeighborhoods\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that if you have any trouble with loading the FeatureCollection
using the technique above, you can follow directions in the Checkpoint
script below to use an existing asset loaded for this exercise.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F50b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{filtering-feature-collections-by-attributes}{%
\subsection{Filtering Feature Collections by
Attributes}\label{filtering-feature-collections-by-attributes}}

\hypertarget{filter-by-geometry-of-another-feature}{%
\subsubsection{Filter by Geometry of Another
Feature}\label{filter-by-geometry-of-another-feature}}

First, let's find the neighborhood associated with USF. Use the first
point you created to find the neighborhood that intersects this point;
filterBounds is the tool that does that, returning a filtered feature.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Filter sfNeighborhoods by USF.  }
\KeywordTok{var}\NormalTok{ usfNeighborhood }\OperatorTok{=}\NormalTok{ sfNeighborhoods}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(usf\_point)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now, filter the blocks layer by USF's neighborhood and visualize it on
the map.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Filter the Census blocks by the boundary of the neighborhood layer.  }
\KeywordTok{var}\NormalTok{ usfTiger }\OperatorTok{=}\NormalTok{ tiger}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(usfNeighborhood)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(usfTiger}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}usf\_Tiger\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\hypertarget{filter-by-feature-attribute-properties}{%
\subsubsection{Filter by Feature (Attribute)
Properties}\label{filter-by-feature-attribute-properties}}

In addition to filtering a FeatureCollection by the location of another
feature, you can also filter it by its properties. First, let's print
the usfTiger variable to the Console and inspect the object.

print(usfTiger);

You can click on the feature collection name in the Console to uncover
more information about the dataset. Click on the columns to learn about
what attribute information is contained in this dataset. You will notice
this feature collection contains information on both housing
(`housing10') and population (`pop10').

Now you will filter for blocks with just the right amount of housing
units. You don't want it too dense, nor do you want too few neighbors.

Filter the blocks to have fewer than 250 housing units.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Filter for census blocks by housing units.  }
\KeywordTok{var}\NormalTok{ housing10\_l250 }\OperatorTok{=}\NormalTok{ usfTiger  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{lt}\NormalTok{(}\StringTok{\textquotesingle{}housing10\textquotesingle{}}\OperatorTok{,} \DecValTok{250}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now filter the already-filtered blocks to have more than 50 housing
units.

var housing10\_g50\_l250 = housing10\_l250.filter(ee.Filter.gt(
`housing10', 50));

Now, let's visualize what this looks like.

Map.addLayer(housing10\_g50\_l250, \{ `color': `Magenta'\}, `housing');

We have combined spatial and attribute information to narrow the set to
only those blocks that meet our criteria of having between 50 and 250
housing units.

\hypertarget{print-feature-attribute-properties-to-console}{%
\subsubsection{Print Feature (Attribute) Properties to
Console}\label{print-feature-attribute-properties-to-console}}

We can print out attribute information about these features. The block
of code below prints out the area of the resultant geometry in square
meters.

var housing\_area = housing10\_g50\_l250.geometry().area();\\
print(`housing\_area:', housing\_area);

The next block of code reduces attribute information and prints out the
mean of the housing10 column.

var housing10\_mean = usfTiger.reduceColumns(\{\\
reducer: ee.Reducer.mean(),\\
selectors: {[}`housing10'{]}\\
\});

print(`housing10\_mean', housing10\_mean);

Both of the above sections of code provide meaningful information about
each feature, but they do not tell us which block is the most green. The
next section will address that question.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F50c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{reducing-images-using-feature-geometry}{%
\subsection{Reducing Images Using Feature
Geometry}\label{reducing-images-using-feature-geometry}}

Now that we have identified the blocks around USF's campus that have the
right housing density, let's find which blocks are the greenest.

The Normalized Difference Vegetation Index (NDVI), presented in detail
in Chap. F2.0, is often used to compare the greenness of pixels in
different locations. Values on land range from 0 to 1, with values
closer to 1 representing healthier and greener vegetation than values
near 0.

\hypertarget{create-an-ndvi-image}{%
\subsubsection{Create an NDVI Image}\label{create-an-ndvi-image}}

The code below imports the Landsat 8 ImageCollection as landsat8. Then,
the code filters for images in 2021. Lastly, the code sorts the images
from 2021 to find the least cloudy day.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the Landsat 8 TOA image collection.  }
\KeywordTok{var}\NormalTok{ landsat8 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_TOA\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Get the least cloudy image in 2015.  }
\KeywordTok{var}\NormalTok{ image }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(  }
\NormalTok{   landsat8  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(usf\_point)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2015{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2015{-}12{-}31\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{sort}\NormalTok{(}\StringTok{\textquotesingle{}CLOUD\_COVER\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{first}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The next section of code assigns the near-infrared band (B5) to variable
nir and assigns the red band (B4) to red. Then the bands are combined
together to compute NDVI as (nir − red)/(nir + red).

var nir = image.select(`B5');\\
var red = image.select(`B4');\\
var ndvi = nir.subtract(red).divide(nir.add(red)).rename(`NDVI');

\hypertarget{clip-the-ndvi-image-to-the-blocks-near-usf}{%
\subsubsection{Clip the NDVI Image to the Blocks Near
USF}\label{clip-the-ndvi-image-to-the-blocks-near-usf}}

Next, you will clip the NDVI layer to only show NDVI over USF's
neighborhood.

The first section of code provides visualization settings.

var ndviParams = \{\\
min: -1,\\
max: 1,\\
palette: {[}`blue', `white', `green'{]}\\
\};

The second block of code clips the image to our filtered housing layer.

var ndviUSFblocks = ndvi.clip(housing10\_g50\_l250);\\
Map.addLayer(ndviUSFblocks, ndviParams, `NDVI image');\\
Map.centerObject(usf\_point, 14);

The NDVI map for all of San Francisco is interesting, and shows
variability across the region. Now, let's compute mean NDVI values for
each block of the city.

\hypertarget{compute-ndvi-statistics-by-block}{%
\subsubsection{Compute NDVI Statistics by
Block}\label{compute-ndvi-statistics-by-block}}

The code below uses the clipped image ndviUSFblocks and computes the
mean NDVI value within each boundary. The scale provides a spatial
resolution for the mean values to be computed on.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Reduce image by feature to compute a statistic e.g. mean, max, min etc.  }
\KeywordTok{var}\NormalTok{ ndviPerBlock }\OperatorTok{=}\NormalTok{ ndviUSFblocks}\OperatorTok{.}\FunctionTok{reduceRegions}\NormalTok{(\{  }
   \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ housing10\_g50\_l250}\OperatorTok{,}  
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:} \DecValTok{30}\OperatorTok{,}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now we'll use Earth Engine to find out which block is greenest.

\hypertarget{export-table-of-ndvi-data-by-block-from-earth-engine-to-google-drive}{%
\subsubsection{Export Table of NDVI Data by Block from Earth Engine to
Google
Drive}\label{export-table-of-ndvi-data-by-block-from-earth-engine-to-google-drive}}

Just as we loaded a feature into Earth Engine, we can export information
from Earth Engine. Here, we will export the NDVI data, summarized by
block, from Earth Engine to a Google Drive space so that we can
interpret it in a program like Google Sheets or Excel.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Get a table of data out of Google Earth Engine.  }
\NormalTok{Export}\OperatorTok{.}\AttributeTok{table}\OperatorTok{.}\FunctionTok{toDrive}\NormalTok{(\{  }
   \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ ndviPerBlock}\OperatorTok{,}  
   \DataTypeTok{description}\OperatorTok{:} \StringTok{\textquotesingle{}NDVI\_by\_block\_near\_USF\textquotesingle{}}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

When you run this code, you will notice that you have the Tasks tab
highlighted on the top right of the Earth Engine Code Editor (Fig.
F5.0.6). You will be prompted to name the directory when exporting the
data.

\begin{figure}

{\centering \includegraphics{./F5/image4.png}

}

\caption{Fig. F5.0.6 Under the Tasks tab, select Run to initiate
download}

\end{figure}

After you run the task, the file will be saved to your Google Drive. You
have now brought a feature into Earth Engine and also exported data from
Earth Engine.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F50d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi}{%
\subsection{Identifying the Block in the Neighborhood Surrounding USF
with the Highest
NDVI}\label{identifying-the-block-in-the-neighborhood-surrounding-usf-with-the-highest-ndvi}}

You are already familiar with filtering datasets by their attributes.
Now you will sort a table and select the first element of the table.

ndviPerBlock = ndviPerBlock.select({[}`blockid10', `mean'{]});\\
print(`ndviPerBlock', ndviPerBlock);\\
var ndviPerBlockSorted = ndviPerBlock.sort(`mean', false);\\
var ndviPerBlockSortedFirst = ee.Feature(ndviPerBlock.sort(`mean',
false) //Sort by NDVI mean in descending order. .first()); //Select the
block with the highest NDVI.\\
print(`ndviPerBlockSortedFirst', ndviPerBlockSortedFirst);

If you expand the feature of ndviPerBlockSortedFirst in the Console, you
will be able to identify the blockid10 value of the greenest block and
the mean NDVI value for that area.

Another way to look at the data is by exporting the data to a table.
Open the table using Google Sheets or Excel. You should see a column
titled ``mean.'' Sort the mean column in descending order from highest
NDVI to lowest NDVI, then use the blockid10 attribute to filter our
feature collection one last time and display the greenest block near
USF.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Now filter by block and show on map!  }
\KeywordTok{var}\NormalTok{ GreenHousing }\OperatorTok{=}\NormalTok{ usfTiger}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{\textquotesingle{}blockid10\textquotesingle{}}\OperatorTok{,}  
\StringTok{\textquotesingle{}\#\#\#\textquotesingle{}}\NormalTok{))}\OperatorTok{;} \CommentTok{//\textless{} Put your id here prepend a 0!  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(GreenHousing}\OperatorTok{,}\NormalTok{ \{   }\StringTok{\textquotesingle{}color\textquotesingle{}}\OperatorTok{:} \StringTok{\textquotesingle{}yellow\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Green Housing!\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F50e. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-17}{%
\subsection*{Conclusion}\label{conclusion-17}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to import features into Earth Engine.
In Sect. 1, you created new features using the geometry tools and loaded
a feature from Earth Engine's Data Catalog. In Sect. 2, you loaded a
shapefile to an Earth Engine asset. In Sect. 3, you filtered feature
collections based on their properties and locations. Finally, in Sects.
4 and 5, you used a feature collection to reduce an image, then exported
the data from Earth Engine. Now you have all the tools you need to load,
filter, and apply features to extract meaningful information from images
using vector features in Earth Engine.

\hypertarget{rastervector-conversions}{%
\section{Raster/Vector Conversions}\label{rastervector-conversions}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-16}{%
\subsubsection*{Author}\label{author-16}}

Keiko Nomura, Samuel Bowers

\hypertarget{overview-18}{%
\subsubsection*{Overview}\label{overview-18}}

The purpose of this chapter is to review methods of converting between
raster and vector data formats, and to understand the circumstances in
which this is useful. By way of example, this chapter focuses on
topographic elevation and forest cover change in Colombia, but note that
these are generic methods that can be applied in a wide variety of
situations.

\hypertarget{learning-outcomes-18}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-18}}

\begin{itemize}
\tightlist
\item
  Understanding raster and vector data in Earth Engine and their
  differing properties.
\item
  Knowing how and why to convert from raster to vector.
\item
  Knowing how and why to convert from vector to raster.
\item
  Write a function and map it over a FeatureCollection.
\end{itemize}

\hypertarget{assumes-you-know-how-to-18}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-18}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Understand distinctions among Image, ImageCollection, Feature and
  FeatureCollection Earth Engine objects (Part F1, Part F2, Part F5).
\item
  Perform basic image analysis: select bands, compute indices, create
  masks (Part F2).
\item
  Perform image morphological operations (Chap. F3.2).
\item
  Understand the filter, map, reduce paradigm (Chap. F4.0).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Use reduceRegions to summarize an image in irregular shapes (Chap.
  F5.0).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-15}{%
\subsection*{Introduction}\label{introduction-15}}

Raster data consists of regularly spaced pixels arranged into rows and
columns, familiar as the format of satellite images. Vector data
contains geometry features (i.e., points, lines, and polygons)
describing locations and areas. Each data format has its advantages, and
both will be encountered as part of GIS operations.

Raster and vector data are commonly combined (e.g., extracting image
information for a given location or clipping an image to an area of
interest); however, there are also situations in which conversion
between the two formats is useful. In making such conversions, it is
important to consider the key advantages of each format. Rasters can
store data efficiently where each pixel has a numerical value, while
vector data can more effectively represent geometric features where
homogenous areas have shared properties. Each format lends itself to
distinctive analytical operations, and combining them can be powerful.

In this exercise, we'll use topographic elevation and forest change
images in Colombia as well as a protected area feature collection to
practice the conversion between raster and vector formats, and to
identify situations in which this is worthwhile.

\hypertarget{raster-to-vector-conversion}{%
\subsection{Raster to Vector
Conversion}\label{raster-to-vector-conversion}}

\hypertarget{raster-to-polygons}{%
\subsubsection{Raster to Polygons}\label{raster-to-polygons}}

In this section we will convert an elevation image (raster) to a feature
collection (vector). We will start by loading the Global
Multi-Resolution Terrain Elevation Data 2010 and the Global
Administrative Unit Layers 2015 dataset to focus on Colombia. The
elevation image is a raster at 7.5 arc-second spatial resolution
containing a continuous measure of elevation in meters in each pixel.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load raster (elevation) and vector (colombia) datasets.  }
\KeywordTok{var}\NormalTok{ elevation }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}USGS/GMTED2010\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}elevation\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ colombia }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(       }\StringTok{\textquotesingle{}FAO/GAUL\_SIMPLIFIED\_500m/2015/level0\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{equals}\NormalTok{(}\StringTok{\textquotesingle{}ADM0\_NAME\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Colombia\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Display elevation image.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(colombia}\OperatorTok{,} \DecValTok{7}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(elevation}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{4000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Elevation\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

When converting an image to a feature collection, we will aggregate the
categorical elevation values into a set of categories to create polygon
shapes of connected pixels with similar elevations. For this exercise,
we will create four zones of elevation by grouping the altitudes to
0-100 m = 0, 100--200 m = 1, 200--500 m = 2, and \textgreater500 m = 3.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Initialize image with zeros and define elevation zones.  }
\KeywordTok{var}\NormalTok{ zones }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\DecValTok{0}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{where}\NormalTok{(elevation}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{100}\NormalTok{)}\OperatorTok{,} \DecValTok{1}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{where}\NormalTok{(elevation}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{200}\NormalTok{)}\OperatorTok{,} \DecValTok{2}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{where}\NormalTok{(elevation}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{500}\NormalTok{)}\OperatorTok{,} \DecValTok{3}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Mask pixels below sea level (\textless{}= 0 m) to retain only land areas.  }
\CommentTok{// Name the band with values 0{-}3 as \textquotesingle{}zone\textquotesingle{}.  }
\NormalTok{zones }\OperatorTok{=}\NormalTok{ zones}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(elevation}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{0}\NormalTok{))}\OperatorTok{.}\FunctionTok{rename}\NormalTok{(}\StringTok{\textquotesingle{}zone\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(zones}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}lime\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}green\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{opacity}\OperatorTok{:} \FloatTok{0.7}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Elevation zones\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We will convert this zonal elevation image in Colombia to polygon
shapes, which is a vector format (termed a FeatureCollection in Earth
Engine), using the ee.Image.reduceToVectors method. This will create
polygons delineating connected pixels with the same value. In doing so,
we will use the same projection and spatial resolution as the image.
Please note that loading the vectorized image in the native resolution
(231.92 m) takes time to execute. For faster visualization, we set a
coarse scale of 1,000 m.

var projection = elevation.projection();\\
var scale = elevation.projection().nominalScale();

var elevationVector = zones.reduceToVectors(\{\\
geometry: colombia.geometry(),\\
crs: projection,\\
scale: 1000, // scale geometryType: `polygon',\\
eightConnected: false,\\
labelProperty: `zone',\\
bestEffort: true,\\
maxPixels: 1e13,\\
tileScale: 3 // In case of error.\\
\});

print(elevationVector.limit(10));

var elevationDrawn = elevationVector.draw(\{\\
color: `black',\\
strokeWidth: 1\\
\});\\
Map.addLayer(elevationDrawn, \{\}, `Elevation zone polygon');

\includegraphics{./F5/image50.png}

\includegraphics{./F5/image33.png}

\includegraphics{./F5/image36.png}

\begin{figure}

{\centering \includegraphics{./F5/image7.png}

}

\caption{Fig. F5.1.1 Raster-based elevation (top left) and zones (top
right), vectorized elevation zones overlaid on the raster (bottom-left)
and vectorized elevation zones only (bottom-right)}

\end{figure}

You may have realized that polygons consist of complex lines, including
some small polygons with just one pixel. That happens when there are no
surrounding pixels of the same elevation zone. You may not need a vector
map with such details---if, for instance, you want to produce a regional
or global map. We can use a morphological reducer focalMode to simplify
the shape by defining a neighborhood size around a pixel. In this
example, we will set the kernel radius as four pixels. This operation
makes the resulting polygons look much smoother, but less precise (Fig.
F5.1.2).

var zonesSmooth = zones.focalMode(4, `square');

zonesSmooth = zonesSmooth.reproject(projection.atScale(scale));

Map.addLayer(zonesSmooth, \{\\
min: 1,\\
max: 3,\\
palette: {[}`yellow', `lime', `green'{]},\\
opacity: 0.7\}, `Elevation zones (smooth)');

var elevationVectorSmooth = zonesSmooth.reduceToVectors(\{\\
geometry: colombia.geometry(),\\
crs: projection,\\
scale: scale,\\
geometryType: `polygon',\\
eightConnected: false,\\
labelProperty: `zone',\\
bestEffort: true,\\
maxPixels: 1e13,\\
tileScale: 3\\
\});

var smoothDrawn = elevationVectorSmooth.draw(\{\\
color: `black',\\
strokeWidth: 1\\
\});\\
Map.addLayer(smoothDrawn, \{\}, `Elevation zone polygon (smooth)');

We can see now that the polygons have more distinct shapes with many
fewer small polygons in the new map (Fig. F5.1.2). It is important to
note that when you use methods like focalMode (or other, similar methods
such as connectedComponents and connectedPixelCount), you need to
reproject according to the original image in order to display properly
with zoom using the interactive Code Editor.

\includegraphics{./F5/image20.png}

\begin{figure}

{\centering \includegraphics{./F5/image37.png}

}

\caption{Fig. F5.1.2 Before (left) and after (right) applying focalMode}

\end{figure}

\hypertarget{raster-to-points}{%
\subsubsection{Raster to Points}\label{raster-to-points}}

Lastly, we will convert a small part of this elevation image into a
point vector dataset. For this exercise, we will use the same example
and build on the code from the previous subsection. This might be useful
when you want to use geospatial data in a tabular format in combination
with other conventional datasets such as economic indicators (Fig.
F5.1.3).

\includegraphics{./F5/image24.png}

\begin{figure}

{\centering \includegraphics{./F5/image11.png}

}

\caption{Fig. F5.1.3 Elevation point values with latitude and longitude}

\end{figure}

The easiest way to do this is to use sample while activating the
geometries parameter. This will extract the points at the centroid of
the elevation pixel.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ geometry }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{([  }
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{89.553}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{0.929}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{89.436}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{0.929}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{89.436}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{0.866}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{89.553}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{0.866}\NormalTok{]}\OperatorTok{,}  
\NormalTok{   [}\OperatorTok{{-}}\FloatTok{89.553}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{0.929}\NormalTok{]  }
\NormalTok{])}\OperatorTok{;}  
  

\CommentTok{// To zoom into the area, un{-}comment and run below  }
\CommentTok{// Map.centerObject(geometry,12);  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(geometry}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Areas to extract points\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ elevationSamples }\OperatorTok{=}\NormalTok{ elevation}\OperatorTok{.}\FunctionTok{sample}\NormalTok{(\{  }
   \DataTypeTok{region}\OperatorTok{:}\NormalTok{ geometry}\OperatorTok{,}  
   \DataTypeTok{projection}\OperatorTok{:}\NormalTok{ projection}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ scale}\OperatorTok{,}  
   \DataTypeTok{geometries}\OperatorTok{:} \KeywordTok{true}\OperatorTok{,}  
\NormalTok{\})}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(elevationSamples}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Points extracted\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Add three properties to the output table:  }
\CommentTok{// \textquotesingle{}Elevation\textquotesingle{}, \textquotesingle{}Longitude\textquotesingle{}, and \textquotesingle{}Latitude\textquotesingle{}.  }
\NormalTok{elevationSamples }\OperatorTok{=}\NormalTok{ elevationSamples}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(feature) \{   }\KeywordTok{var}\NormalTok{ geom }\OperatorTok{=}\NormalTok{ feature}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{.}\FunctionTok{coordinates}\NormalTok{()}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Feature}\NormalTok{(}\KeywordTok{null}\OperatorTok{,}\NormalTok{ \{       }\StringTok{\textquotesingle{}Elevation\textquotesingle{}}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(feature}\OperatorTok{.}\FunctionTok{get}\NormalTok{(           }\StringTok{\textquotesingle{}elevation\textquotesingle{}}\NormalTok{))}\OperatorTok{,}       \StringTok{\textquotesingle{}Long\textquotesingle{}}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(geom}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{))}\OperatorTok{,}       \StringTok{\textquotesingle{}Lat\textquotesingle{}}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(geom}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{1}\NormalTok{))  }
\NormalTok{   \})}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Export as CSV.  }
\NormalTok{Export}\OperatorTok{.}\AttributeTok{table}\OperatorTok{.}\FunctionTok{toDrive}\NormalTok{(\{  }
   \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ elevationSamples}\OperatorTok{,}  
   \DataTypeTok{description}\OperatorTok{:} \StringTok{\textquotesingle{}extracted\_points\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{fileFormat}\OperatorTok{:} \StringTok{\textquotesingle{}CSV\textquotesingle{}}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We can also extract sample points per elevation zone. Below is an
example of extracting 10 randomly selected points per elevation zone
(Fig. F5.1.4). You can also set different values for each zone using
classValues and classPoints parameters to modify the sampling intensity
in each class. This may be useful, for instance, to generate point
samples for a validation effort.

var elevationSamplesStratified = zones.stratifiedSample(\{\\
numPoints: 10,\\
classBand: `zone',\\
region: geometry,\\
scale: scale,\\
projection: projection,\\
geometries: true\\
\});

Map.addLayer(elevationSamplesStratified, \{\}, `Stratified samples');

\begin{figure}

{\centering \includegraphics{./F5/image23.png}

}

\caption{Fig. F5.1.4 Stratified sampling over different elevation zones}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F51a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\#\#\#3. A More Complex Example

In this section we'll use two global datasets, one to represent raster
formats and the other vectors:

\begin{itemize}
\tightlist
\item
  The Global Forest Change (GFC) dataset: a raster dataset describing
  global tree cover and change for 2001--present.
\item
  The World Protected Areas Database: a vector database of global
  protected areas.
\end{itemize}

The objective will be to combine these two datasets to quantify rates of
deforestation in protected areas in the ``arc of deforestation'' of the
Colombian Amazon. The datasets can be loaded into Earth Engine with the
following code:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Read input data.  }
\CommentTok{// Note: these datasets are periodically updated.  }
\CommentTok{// Consider searching the Data Catalog for newer versions.  }
\KeywordTok{var}\NormalTok{ gfc }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}UMD/hansen/global\_forest\_change\_2020\_v1\_8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ wdpa }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}WCMC/WDPA/current/polygons\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Print assets to show available layers and properties.  }
\FunctionTok{print}\NormalTok{(gfc)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(wdpa}\OperatorTok{.}\FunctionTok{limit}\NormalTok{(}\DecValTok{10}\NormalTok{))}\OperatorTok{;} \CommentTok{// Show first 10 records.}

\NormalTok{The GFC }\FunctionTok{dataset}\NormalTok{ (first presented }\KeywordTok{in}\NormalTok{ detail }\KeywordTok{in}\NormalTok{ Chap}\OperatorTok{.} \AttributeTok{F1}\OperatorTok{.}\DecValTok{1}\NormalTok{) is a }\BuiltInTok{global}\NormalTok{ set }\KeywordTok{of}\NormalTok{ rasters that quantify tree cover and change }\ControlFlowTok{for}\NormalTok{ the period beginning }\KeywordTok{in} \FloatTok{2001.}\NormalTok{ We’ll use a single image }\ImportTok{from} \KeywordTok{this}\NormalTok{ dataset}\OperatorTok{:}

\OperatorTok{*}   \StringTok{\textquotesingle{}lossyear\textquotesingle{}}\OperatorTok{:}\NormalTok{ a categorical raster }\KeywordTok{of}\NormalTok{ forest }\FunctionTok{loss}\NormalTok{ (}\DecValTok{1}\ErrorTok{–20}\OperatorTok{,}\NormalTok{ corresponding to deforestation }\ControlFlowTok{for}\NormalTok{ the period }\DecValTok{2001}\ErrorTok{–2020}\NormalTok{)}\OperatorTok{,}\NormalTok{ and }\DecValTok{0} \ControlFlowTok{for}\NormalTok{ no change}

\NormalTok{The World Database on Protected }\FunctionTok{Areas}\NormalTok{ (WDPA) is a harmonized dataset }\KeywordTok{of} \BuiltInTok{global}\NormalTok{ terrestrial and marine }\KeywordTok{protected}\NormalTok{ area locations}\OperatorTok{,}\NormalTok{ along }\ControlFlowTok{with}\NormalTok{ details on the classification and management }\KeywordTok{of}\NormalTok{ each}\OperatorTok{.} \AttributeTok{In}\NormalTok{ addition to }\KeywordTok{protected}\NormalTok{ area outlines}\OperatorTok{,}\NormalTok{ we’ll use two fields }\ImportTok{from} \KeywordTok{this}\NormalTok{ database}\OperatorTok{:}

\OperatorTok{*}   \StringTok{\textquotesingle{}NAME\textquotesingle{}}\NormalTok{’}\OperatorTok{:}\NormalTok{ the name }\KeywordTok{of}\NormalTok{ each }\KeywordTok{protected}\NormalTok{ area}
\OperatorTok{*}\NormalTok{   ‘WDPA\_PID’}\OperatorTok{:}\NormalTok{ a unique numerical ID }\ControlFlowTok{for}\NormalTok{ each }\KeywordTok{protected}\NormalTok{ area}

\NormalTok{To begin }\ControlFlowTok{with}\OperatorTok{,}\NormalTok{ we’ll focus on forest change dynamics }\KeywordTok{in}\NormalTok{ ‘La Paya’}\OperatorTok{,}\NormalTok{ a small }\KeywordTok{protected}\NormalTok{ area }\KeywordTok{in}\NormalTok{ the Colombian Amazon}\OperatorTok{.} \AttributeTok{We’ll}\NormalTok{ first visualize these data using the paint command}\OperatorTok{,}\NormalTok{ which is discussed }\KeywordTok{in}\NormalTok{ more detail }\KeywordTok{in}\NormalTok{ Chap}\OperatorTok{.} \AttributeTok{F5}\OperatorTok{.}\DecValTok{3}\OperatorTok{:}

\CommentTok{// Display deforestation.  }
\KeywordTok{var}\NormalTok{ deforestation }\OperatorTok{=}\NormalTok{ gfc}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}lossyear\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(deforestation}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}orange\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Deforestation raster\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Display WDPA data.  }
\KeywordTok{var}\NormalTok{ protectedArea }\OperatorTok{=}\NormalTok{ wdpa}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{equals}\NormalTok{(}\StringTok{\textquotesingle{}NAME\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}La Paya\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Display protected area as an outline (see F5.3 for paint()).  }
\KeywordTok{var}\NormalTok{ protectedAreaOutline }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{  }
   \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ protectedArea}\OperatorTok{,}  
   \DataTypeTok{color}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{width}\OperatorTok{:} \DecValTok{3}  
\NormalTok{\})}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(protectedAreaOutline}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:} \StringTok{\textquotesingle{}white\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Protected area\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Set up map display.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(protectedArea)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{\textquotesingle{}SATELLITE\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

This will display the boundary of the La Paya protected area and
deforestation in the region (Fig. F5.1.5).

\begin{figure}

{\centering \includegraphics{./F5/image55.png}

}

\caption{Fig. F5.1.5 View of the La Paya protected area in the Colombian
Amazon (in white), and deforestation over the period 2001--2020 (in
yellows and reds, with darker colors indicating more recent changes)}

\end{figure}

We can use Earth Engine to convert the deforestation raster to a set of
polygons. The deforestation data are appropriate for this transformation
as each deforestation event is labeled categorically by year, and change
events are spatially contiguous. This is performed in Earth Engine using
the ee.Image.reduceToVectors method, as described earlier in this
section.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Convert from a deforestation raster to vector.  }
\KeywordTok{var}\NormalTok{ deforestationVector }\OperatorTok{=}\NormalTok{ deforestation}\OperatorTok{.}\FunctionTok{reduceToVectors}\NormalTok{(\{  }
   \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ deforestation}\OperatorTok{.}\FunctionTok{projection}\NormalTok{()}\OperatorTok{.}\FunctionTok{nominalScale}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{geometry}\OperatorTok{:}\NormalTok{ protectedArea}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{labelProperty}\OperatorTok{:} \StringTok{\textquotesingle{}lossyear\textquotesingle{}}\OperatorTok{,} \CommentTok{// Label polygons with a change year.   maxPixels: 1e13  }
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Count the number of individual change events  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Number of change events:\textquotesingle{}}\OperatorTok{,}\NormalTok{ deforestationVector}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}  
  
\CommentTok{// Display deforestation polygons. Color outline by change year.  }
\KeywordTok{var}\NormalTok{ deforestationVectorOutline }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{  }
   \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ deforestationVector}\OperatorTok{,}  
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}lossyear\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{width}\OperatorTok{:} \DecValTok{1}  
\NormalTok{\})}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(deforestationVectorOutline}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}orange\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Deforestation vector\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Fig. F5.1.6 shows a comparison of the raster versus vector
representations of deforestation within the protected area.

\includegraphics{./F5/image42.png}

\begin{figure}

{\centering \includegraphics{./F5/image13.png}

}

\caption{Fig. F5.1.6 Raster (left) versus vector (right) representations
of deforestation data of the La Paya protected area}

\end{figure}

Having converted from raster to vector, a new set of operations becomes
available for post-processing the deforestation data. We might, for
instance, be interested in the number of individual change events each
year (Fig. F5.1.7):

var chart = ui.Chart.feature\\
.histogram(\{\\
features: deforestationVector,\\
property: `lossyear' \})\\
.setOptions(\{\\
hAxis: \{\\
title: `Year' \},\\
vAxis: \{\\
title: `Number of deforestation events' \},\\
legend: \{\\
position: `none' \}\\
\});print(chart);

\begin{figure}

{\centering \includegraphics{./F5/image15.png}

}

\caption{Fig. F5.1.7 Plot of the number of deforestation events in La
Paya for the years 2001--2020}

\end{figure}

There might also be interest in generating point locations for
individual change events (e.g., to aid a field campaign):

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Generate deforestation point locations.  }
\KeywordTok{var}\NormalTok{ deforestationCentroids }\OperatorTok{=}\NormalTok{ deforestationVector}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(feat) \{   }\ControlFlowTok{return}\NormalTok{ feat}\OperatorTok{.}\FunctionTok{centroid}\NormalTok{()}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(deforestationCentroids}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}darkblue\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Deforestation centroids\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The vector format allows for easy filtering to only deforestation events
of interest, such as only the largest deforestation events:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Add a new property to the deforestation FeatureCollection  }
\CommentTok{// describing the area of the change polygon.  }
\NormalTok{deforestationVector }\OperatorTok{=}\NormalTok{ deforestationVector}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(feat) \{   }\ControlFlowTok{return}\NormalTok{ feat}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{\textquotesingle{}area\textquotesingle{}}\OperatorTok{,}\NormalTok{ feat}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{.}\FunctionTok{area}\NormalTok{(\{  }
       \DataTypeTok{maxError}\OperatorTok{:} \DecValTok{10}\NormalTok{   \})}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(}\DecValTok{10000}\NormalTok{))}\OperatorTok{;} \CommentTok{// Convert m\^{}2 to hectare.  }
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Filter the deforestation FeatureCollection for only large{-}scale (\textgreater{}10 ha) changes  }
\KeywordTok{var}\NormalTok{ deforestationLarge }\OperatorTok{=}\NormalTok{ deforestationVector}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(   }\StringTok{\textquotesingle{}area\textquotesingle{}}\OperatorTok{,} \DecValTok{10}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Display deforestation area outline by year.  }
\KeywordTok{var}\NormalTok{ deforestationLargeOutline }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{  }
   \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ deforestationLarge}\OperatorTok{,}  
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}lossyear\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{width}\OperatorTok{:} \DecValTok{1}  
\NormalTok{\})}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(deforestationLargeOutline}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}orange\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Deforestation (\textgreater{}10 ha)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F51b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{raster-properties-to-vector-fields}{%
\subsubsection{Raster Properties to Vector
Fields}\label{raster-properties-to-vector-fields}}

Sometimes we want to extract information from a raster to be included in
an existing vector dataset. An example might be estimating a
deforestation rate for a set of protected areas. Rather than perform
this task on a case-by-case basis, we can attach information generated
from an image as a property of a feature.

The following script shows how this can be used to quantify a
deforestation rate for a set of protected areas in the Colombian Amazon.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load required datasets.  }
\KeywordTok{var}\NormalTok{ gfc }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}UMD/hansen/global\_forest\_change\_2020\_v1\_8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ wdpa }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}WCMC/WDPA/current/polygons\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Display deforestation.  }
\KeywordTok{var}\NormalTok{ deforestation }\OperatorTok{=}\NormalTok{ gfc}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}lossyear\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(deforestation}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}orange\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Deforestation raster\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Select protected areas in the Colombian Amazon.  }
\KeywordTok{var}\NormalTok{ amazonianProtectedAreas }\OperatorTok{=}\NormalTok{ [   }\StringTok{\textquotesingle{}Cordillera de los Picachos\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}La Paya\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Nukak\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}Serrania de Chiribiquete\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}Sierra de la Macarena\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Tinigua\textquotesingle{}}  
\NormalTok{]}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ wdpaSubset }\OperatorTok{=}\NormalTok{ wdpa}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{inList}\NormalTok{(}\StringTok{\textquotesingle{}NAME\textquotesingle{}}\OperatorTok{,}  
\NormalTok{   amazonianProtectedAreas))}\OperatorTok{;}  
  
\CommentTok{// Display protected areas as an outline.  }
\KeywordTok{var}\NormalTok{ protectedAreasOutline }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{  }
   \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ wdpaSubset}\OperatorTok{,}  
   \DataTypeTok{color}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{width}\OperatorTok{:} \DecValTok{1}  
\NormalTok{\})}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(protectedAreasOutline}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:} \StringTok{\textquotesingle{}white\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Amazonian protected areas\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Set up map display.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(wdpaSubset)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{\textquotesingle{}SATELLITE\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ scale }\OperatorTok{=}\NormalTok{ deforestation}\OperatorTok{.}\FunctionTok{projection}\NormalTok{()}\OperatorTok{.}\FunctionTok{nominalScale}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Use \textquotesingle{}reduceRegions\textquotesingle{} to sum together pixel areas in each protected area.  }
\NormalTok{wdpaSubset }\OperatorTok{=}\NormalTok{ deforestation}\OperatorTok{.}\FunctionTok{gte}\NormalTok{(}\DecValTok{1}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{multiply}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\FunctionTok{pixelArea}\NormalTok{()}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(}\DecValTok{10000}\NormalTok{))}\OperatorTok{.}\FunctionTok{reduceRegions}\NormalTok{(\{  }
       \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ wdpaSubset}\OperatorTok{,}  
       \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{sum}\NormalTok{()}\OperatorTok{.}\FunctionTok{setOutputs}\NormalTok{([           }\StringTok{\textquotesingle{}deforestation\_area\textquotesingle{}}\NormalTok{])}\OperatorTok{,}  
       \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ scale  }
\NormalTok{   \})}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(wdpaSubset)}\OperatorTok{;} \CommentTok{// Note the new \textquotesingle{}deforestation\_area\textquotesingle{} property.}

\NormalTok{The output }\KeywordTok{of} \KeywordTok{this}\NormalTok{ script is an estimate }\KeywordTok{of}\NormalTok{ deforested area }\KeywordTok{in}\NormalTok{ hectares }\ControlFlowTok{for}\NormalTok{ each reserve}\OperatorTok{.} \AttributeTok{However}\OperatorTok{,} \ImportTok{as}\NormalTok{ reserve sizes vary substantially by area}\OperatorTok{,}\NormalTok{ we can normalize by the total area }\KeywordTok{of}\NormalTok{ each reserve to quantify rates }\KeywordTok{of}\NormalTok{ change}\OperatorTok{.}

\CommentTok{// Normalize by area.  }
\NormalTok{wdpaSubset }\OperatorTok{=}\NormalTok{ wdpaSubset}\OperatorTok{.}\FunctionTok{map}\NormalTok{(   }\KeywordTok{function}\NormalTok{(feat) \{       }\ControlFlowTok{return}\NormalTok{ feat}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{\textquotesingle{}deforestation\_rate\textquotesingle{}}\OperatorTok{,}\NormalTok{           ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(feat}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}deforestation\_area\textquotesingle{}}\NormalTok{))  }
           \OperatorTok{.}\FunctionTok{divide}\NormalTok{(feat}\OperatorTok{.}\FunctionTok{area}\NormalTok{()}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(}\DecValTok{10000}\NormalTok{)) }\CommentTok{// m2 to ha           .divide(20) // number of years           .multiply(100)); // to percentage points   \});// Print to identify rates of change per protected area.  }
\CommentTok{// Which has the fastest rate of loss?  }
\FunctionTok{print}\NormalTok{(wdpaSubset}\OperatorTok{.}\FunctionTok{reduceColumns}\NormalTok{(\{  }
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{toList}\NormalTok{()}\OperatorTok{.}\FunctionTok{repeat}\NormalTok{(}\DecValTok{2}\NormalTok{)}\OperatorTok{,}  
   \DataTypeTok{selectors}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}NAME\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}deforestation\_rate\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F51c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{vector-to-raster-conversion}{%
\subsection{Vector-to-Raster
Conversion}\label{vector-to-raster-conversion}}

In Sect. 1, we used the protected area feature collection as its
original vector format. In this section, we will rasterize the protected
area polygons to produce a mask and use this to assess rates of forest
change.

\hypertarget{polygons-to-a-mask}{%
\subsubsection{Polygons to a Mask}\label{polygons-to-a-mask}}

The most common operation to convert from vector to raster is the
production of binary image masks, describing whether a pixel intersects
a line or falls within a polygon. To convert from vector to a raster
mask, we can use the ee.FeatureCollection.reduceToImage method. Let's
continue with our example of the WDPA database and Global Forest Change
data from the previous section:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load required datasets.  }
\KeywordTok{var}\NormalTok{ gfc }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}UMD/hansen/global\_forest\_change\_2020\_v1\_8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ wdpa }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}WCMC/WDPA/current/polygons\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Get deforestation.  }
\KeywordTok{var}\NormalTok{ deforestation }\OperatorTok{=}\NormalTok{ gfc}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}lossyear\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Generate a new property called \textquotesingle{}protected\textquotesingle{} to apply to the output mask.  }
\KeywordTok{var}\NormalTok{ wdpa }\OperatorTok{=}\NormalTok{ wdpa}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(feat) \{   }\ControlFlowTok{return}\NormalTok{ feat}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{\textquotesingle{}protected\textquotesingle{}}\OperatorTok{,} \DecValTok{1}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\CommentTok{// Rasterize using the new property.  }
\CommentTok{// unmask() sets areas outside protected area polygons to 0.  }
\KeywordTok{var}\NormalTok{ wdpaMask }\OperatorTok{=}\NormalTok{ wdpa}\OperatorTok{.}\FunctionTok{reduceToImage}\NormalTok{([}\StringTok{\textquotesingle{}protected\textquotesingle{}}\NormalTok{]}\OperatorTok{,}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{first}\NormalTok{())  }
   \OperatorTok{.}\FunctionTok{unmask}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Center on Colombia.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\OperatorTok{{-}}\DecValTok{75}\OperatorTok{,} \DecValTok{3}\OperatorTok{,} \DecValTok{6}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Display on map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(wdpaMask}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Protected areas (mask)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We can use this mask to, for example, highlight only deforestation that
occurs within a protected area using logical operations:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Set the deforestation layer to 0 where outside a protected area.  }
\KeywordTok{var}\NormalTok{ deforestationProtected }\OperatorTok{=}\NormalTok{ deforestation}\OperatorTok{.}\FunctionTok{where}\NormalTok{(wdpaMask}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{,} \DecValTok{0}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Update mask to hide where deforestation layer = 0  }
\KeywordTok{var}\NormalTok{ deforestationProtected }\OperatorTok{=}\NormalTok{ deforestationProtected  }
   \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(deforestationProtected}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{0}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Display deforestation in protected areas  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(deforestationProtected}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}yellow\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}orange\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Deforestation protected\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In the above example we generated a simple binary mask, but
reduceToImage can also preserve a numerical property of the input
polygons. For example, we might want to be able to determine which
protected area each pixel represents. In this case, we can produce an
image with the unique ID of each protected area:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Produce an image with unique ID of protected areas.  }
\KeywordTok{var}\NormalTok{ wdpaId }\OperatorTok{=}\NormalTok{ wdpa}\OperatorTok{.}\FunctionTok{reduceToImage}\NormalTok{([}\StringTok{\textquotesingle{}WDPAID\textquotesingle{}}\NormalTok{]}\OperatorTok{,}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{first}\NormalTok{())}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(wdpaId}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{100000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Protected area ID\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

This output can be useful when performing large-scale raster operations,
such as efficiently calculating deforestation rates for multiple
protected areas.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F51d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{a-more-complex-example}{%
\subsubsection{A More Complex Example}\label{a-more-complex-example}}

The reduceToImage method is not the only way to convert a feature
collection to an image. We will create a distance image layer from the
boundary of the protected area using distance. For this example, we
return to the La Paya protected area explored in Sect. 1.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load required datasets.  }
\KeywordTok{var}\NormalTok{ gfc }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}UMD/hansen/global\_forest\_change\_2020\_v1\_8\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ wdpa }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}WCMC/WDPA/current/polygons\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Select a single protected area.  }
\KeywordTok{var}\NormalTok{ protectedArea }\OperatorTok{=}\NormalTok{ wdpa}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{equals}\NormalTok{(}\StringTok{\textquotesingle{}NAME\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}La Paya\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Maximum distance in meters is set in the brackets.  }
\KeywordTok{var}\NormalTok{ distance }\OperatorTok{=}\NormalTok{ protectedArea}\OperatorTok{.}\FunctionTok{distance}\NormalTok{(}\DecValTok{1000000}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(distance}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20000}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}grey\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}black\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{opacity}\OperatorTok{:} \FloatTok{0.6}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Distance\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(protectedArea)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We can also show the distance inside and outside of the boundary by
using the rasterized protected area (Fig. F5.1.8).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Produce a raster of inside/outside the protected area.  }
\KeywordTok{var}\NormalTok{ protectedAreaRaster }\OperatorTok{=}\NormalTok{ protectedArea}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(feat) \{   }\ControlFlowTok{return}\NormalTok{ feat}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{\textquotesingle{}protected\textquotesingle{}}\OperatorTok{,} \DecValTok{1}\NormalTok{)}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{.}\FunctionTok{reduceToImage}\NormalTok{([}\StringTok{\textquotesingle{}protected\textquotesingle{}}\NormalTok{]}\OperatorTok{,}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{first}\NormalTok{())}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(distance}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(protectedAreaRaster)}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Distance inside protected area\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(distance}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(protectedAreaRaster}\OperatorTok{.}\FunctionTok{unmask}\NormalTok{()  }
\OperatorTok{.}\FunctionTok{not}\NormalTok{())}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{20000}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Distance outside protected area\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\includegraphics{./F5/image56.png}

\includegraphics{./F5/image9.png}

\begin{figure}

{\centering \includegraphics{./F5/image25.png}

}

\caption{Fig. F5.1.8 Distance from the La Paya boundary (left), distance
within the La Paya (middle), and distance outside the La Paya (right)}

\end{figure}

Sometimes it makes sense to work with objects in raster imagery. This is
an unusual case of vector-like operations conducted with raster data.
There is a good reason for this where the vector equivalent would be
computationally burdensome.

An example of this is estimating deforestation rates by distance to the
edge of the protected area, as it is common that rates of change will be
higher at the boundary of a protected area. We will create a distance
raster with three zones from the La Paya boundary (\textgreater1 km,
\textgreater2 km, \textgreater3 km, and \textgreater4 km) and to
estimate the deforestation by distance from the boundary (Fig. F5.1.9).

var distanceZones = ee.Image(0)\\
.where(distance.gt(0), 1)\\
.where(distance.gt(1000), 2)\\
.where(distance.gt(3000), 3)\\
.updateMask(distance.lte(5000));

Map.addLayer(distanceZones, \{\}, `Distance zones');

var deforestation = gfc.select(`loss');\\
var deforestation1km = deforestation.updateMask(distanceZones.eq(1));\\
var deforestation3km = deforestation.updateMask(distanceZones.lte(2));\\
var deforestation5km = deforestation.updateMask(distanceZones.lte(3));

Map.addLayer(deforestation1km, \{\\
min: 0,\\
max: 1\}, `Deforestation within a 1km buffer');\\
Map.addLayer(deforestation3km, \{\\
min: 0,\\
max: 1,\\
opacity: 0.5\}, `Deforestation within a 3km buffer');\\
Map.addLayer(deforestation5km, \{\\
min: 0,\\
max: 1,\\
opacity: 0.5\}, `Deforestation within a 5km buffer');

\includegraphics{./F5/image22.png}

\includegraphics{./F5/image6.png}

\includegraphics{./F5/image21.png}

\begin{figure}

{\centering \includegraphics{./F5/image26.png}

}

\caption{Fig. F5.1.9 Distance zones (top left) and deforestation by zone
(\textless1 km, \textless3 km, and \textless5 km)}

\end{figure}

Lastly, we can estimate the deforestation area within 1 km of the
protected area but only outside of the boundary.

var deforestation1kmOutside = deforestation1km\\
.updateMask(protectedAreaRaster.unmask().not());

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Get the value of each pixel in square meters  }
\CommentTok{// and divide by 10000 to convert to hectares.  }
\KeywordTok{var}\NormalTok{ deforestation1kmOutsideArea }\OperatorTok{=}\NormalTok{ deforestation1kmOutside}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\DecValTok{1}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{multiply}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\FunctionTok{pixelArea}\NormalTok{())}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(}\DecValTok{10000}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// We need to set a larger geometry than the protected area  }
\CommentTok{// for the geometry parameter in reduceRegion().  }
\KeywordTok{var}\NormalTok{ deforestationEstimate }\OperatorTok{=}\NormalTok{ deforestation1kmOutsideArea  }
   \OperatorTok{.}\FunctionTok{reduceRegion}\NormalTok{(\{  }
       \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{sum}\NormalTok{()}\OperatorTok{,}  
       \DataTypeTok{geometry}\OperatorTok{:}\NormalTok{ protectedArea}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{.}\FunctionTok{buffer}\NormalTok{(}\DecValTok{1000}\NormalTok{)}\OperatorTok{,}  
       \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ deforestation}\OperatorTok{.}\FunctionTok{projection}\NormalTok{()}\OperatorTok{.}\FunctionTok{nominalScale}\NormalTok{()  }
\NormalTok{   \})}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Deforestation within a 1km buffer outside the protected area (ha)\textquotesingle{}}\OperatorTok{,}  
\NormalTok{   deforestationEstimate)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F51e. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-18}{%
\subsection*{Conclusion}\label{conclusion-18}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to convert raster to vector and vice
versa. More importantly, you now have a better understanding of why and
when such conversions are useful. Our examples should give you practical
applications and ideas for using these techniques.

\hypertarget{zonal-statistics}{%
\section{Zonal Statistics}\label{zonal-statistics}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-17}{%
\subsubsection*{Author}\label{author-17}}

Sara Winsemius and Justin Braaten

\hypertarget{overview-19}{%
\subsubsection*{Overview}\label{overview-19}}

The purpose of this chapter is to extract values from rasters for
intersecting points or polygons. We will lay out the process and a
function to calculate zonal statistics, which includes optional
parameters to modify the function, and then apply the process to three
examples using different raster datasets and combinations of parameters.

\hypertarget{learning-outcomes-19}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-19}}

\begin{itemize}
\tightlist
\item
  Buffering points as square or circular regions.
\item
  Writing and applying functions with optional parameters.
\item
  Learning what zonal statistics are and how to use reducers.
\item
  Exporting computation results to a table.
\item
  Copying properties from one image to another.
\end{itemize}

\hypertarget{assumes-you-know-how-to-19}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-19}}

\begin{itemize}
\tightlist
\item
  Recognize similarities and differences among Landsat 5, 7, and 8
  spectral bands (Part F1, Part F2, Part F3).
\item
  Understand distinctions among Image, ImageCollection, Feature and
  FeatureCollection Earth Engine objects (Part F1, Part F2, Part F5).
\item
  Use drawing tools to create points, lines, and polygons (Chap. F2.1).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Mask cloud, cloud shadow, snow/ice, and other undesired pixels (Chap.
  F4.3).
\item
  Export calculated data to tables with Tasks (Chap. F5.0).
\item
  Understand the differences between raster and vector data (Chap. F5.0,
  Chap. F5.1).
\item
  Write a function and map it over a FeatureCollection (Chap. F5.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-16}{%
\subsection*{Introduction}\label{introduction-16}}

Anyone working with field data collected at plots will likely need to
summarize raster-based data associated with those plots. For instance,
they need to know the Normalized Difference Vegetation Index (NDVI),
precipitation, or elevation for each plot (or surrounding region).
Calculating statistics from a raster within given regions is called
zonal statistics. Zonal statistics were calculated in Chaps. F5.0 and
F5.1 using ee.Image.ReduceRegions. Here, we present a more general
approach to calculating zonal statistics with a custom function that
works for both ee.Image and ee.ImageCollection objects. In addition to
its flexibility, the reduction method used here is less prone to
``Computed value is too large'' errors that can occur when using
ReduceRegions with very large or complex ee.FeatureCollection object
inputs.

The zonal statistics function in this chapter works for an Image or an
ImageCollection. Running the function over an ImageCollection will
produce a table with values from each image in the collection per point.
Image collections can be processed before extraction as needed---for
example, by masking clouds from satellite imagery or by constraining the
dates needed for a particular research question. In this tutorial, the
data extracted from rasters are exported to a table for analysis, where
each row of the table corresponds to a unique point-image combination.

In fieldwork, researchers often work with plots, which are commonly
recorded as polygon files or as a center point with a set radius. It is
rare that plots will be set directly in the center of pixels from your
desired raster dataset, and many field GPS units have positioning
errors. Because of these issues, it may be important to use a statistic
of adjacent pixels (as described in Chap. F3.2) to estimate the central
value in what's often called a neighborhood mean or focal mean (Cansler
and McKenzie 2012, Miller and Thode 2007).

To choose the size of your neighborhood, you will need to consider your
research questions, the spatial resolution of the dataset, the size of
your field plot, and the error from your GPS. For example, the raster
value extracted for randomly placed 20 m diameter plots would likely
merit use of a neighborhood mean when using Sentinel-2 or Landsat 8---at
10 m and 30 m spatial resolution, respectively---while using a thermal
band from MODIS (Moderate Resolution Imaging Spectroradiometer) at 1000
m may not. While much of this tutorial is written with plot points and
buffers in mind, a polygon asset with predefined regions will serve the
same purpose.

\hypertarget{functions-1}{%
\subsection{Functions}\label{functions-1}}

Two functions are provided; copy and paste them into your script:

\begin{itemize}
\tightlist
\item
  A function to generate circular or square regions from buffered points
\item
  A function to extract image pixel neighborhood statistics for a given
  region
\end{itemize}

\hypertarget{function-bufferpointsradius-bounds}{%
\subsubsection{Function: bufferPoints(radius,
bounds)}\label{function-bufferpointsradius-bounds}}

Our first function, bufferPoints, returns a function for adding a buffer
to points and optionally transforming to rectangular bounds (see Table
F5.2.1).

Table F5.2.1 Parameters for bufferPoints

Parameter

Type

Description

radius

Number

buffer radius (m).

{[}bounds=false{]}

Boolean

An optional flag indicating whether to transform buffered point (i.e., a
circle) to square bounds.

function bufferPoints(radius, bounds) \{ return function(pt) \{\\
pt = ee.Feature(pt); return bounds ? pt.buffer(radius).bounds() :
pt.buffer(\\
radius);\\
\};\\
\}

\hypertarget{function-zonalstatsfc-params}{%
\subsubsection{Function: zonalStats(fc,
params)}\label{function-zonalstatsfc-params}}

The second function, zonalStats, reduces images in an ImageCollection by
regions defined in a FeatureCollection. Note that reductions can return
null statistics that you might want to filter out of the resulting
feature collection. Null statistics occur when there are no valid pixels
intersecting the region being reduced. This situation can be caused by
points that are outside of an image or in regions that are masked for
quality or clouds.

This function is written to include many optional parameters (see Table
F5.2.2). Look at the function carefully and note how it is written to
include defaults that make it easy to apply the basic function while
allowing customization.

Table F5.2.2 Parameters for zonalStats

Parameter

Type

Description

ic

ee.ImageCollection

Image collection from which to extract values.

fc

ee.FeatureCollection

Feature collection that provides regions/zones by which to reduce image
pixels.

{[}params{]}

Object

An optional Object that provides function arguments.

{[}params.reducer=ee.Reducer.mean(){]}

ee.Reducer

The reducer to apply. Optional.

{[}params.scale=null{]}

Number

A nominal scale in meters of the projection to work in. If null, the
native nominal image scale is used. Optional.

{[}params.crs=null{]}

String

The projection to work in. If null, the native image Coordinate
Reference System (CRS) is used. Optional.

{[}params.bands=null{]}

Array

A list of image band names for which to reduce values. If null, all
bands will be reduced. Band names define column names in the resulting
reduction table. Optional.

{[}params.bandsRename=null{]}

Array

A list of desired image band names. The length and order must correspond
to the params.bands list. If null, band names will be unchanged. Band
names define column names in the resulting reduction table. Optional.

{[}params.imgProps=null{]}

Array

A list of image properties to include in the table of region reduction
results. If null, all image properties are included. Optional.

{[}params.imgPropsRename=null{]}

Array

A list of image property names to replace those provided by
params.imgProps. The length and order must match the params.imgProps
entries. Optional.

{[}params.datetimeName='datetime{]}

String

The desired name of the datetime field. The datetime refers to the
`system:time\_start' value of the ee.Image being reduced. Optional.

{[}params.datetimeFormat='YYYY-MM-dd HH:mm:ss{]}

String

The desired datetime format. Use ISO 8601 data string standards. The
datetime string is derived from the `system:time\_start' value of the
ee.Image being reduced. Optional.

function zonalStats(ic, fc, params) \{ // Initialize internal params
dictionary. var \_params = \{\\
reducer: ee.Reducer.mean(),\\
scale: null,\\
crs: null,\\
bands: null,\\
bandsRename: null,\\
imgProps: null,\\
imgPropsRename: null,\\
datetimeName: `datetime',\\
datetimeFormat: `YYYY-MM-dd HH:mm:ss' \}; // Replace initialized params
with provided params. if (params) \{ for (var param in params) \{\\
\_params{[}param{]} = params{[}param{]} \textbar\textbar{}
\_params{[}param{]};\\
\}\\
\} // Set default parameters based on an image representative. var
imgRep = ic.first(); var nonSystemImgProps = ee.Feature(null)\\
.copyProperties(imgRep).propertyNames(); if (!\_params.bands)
\_params.bands = imgRep.bandNames(); if (!\_params.bandsRename)
\_params.bandsRename = \_params.bands; if (!\_params.imgProps)
\_params.imgProps = nonSystemImgProps; if (!\_params.imgPropsRename)
\_params.imgPropsRename = \_params\\
.imgProps; // Map the reduceRegions function over the image collection.
var results = ic.map(function(img) \{ // Select bands (optionally
rename), set a datetime \& timestamp property. img =
ee.Image(img.select(\_params.bands, \_params\\
.bandsRename)) // Add datetime and timestamp features.
.set(\_params.datetimeName, img.date().format(\\
\_params.datetimeFormat)) .set(`timestamp',
img.get(`system:time\_start')); // Define final image property
dictionary to set in output features. var propsFrom =
ee.List(\_params.imgProps) .cat(ee.List({[}\_params.datetimeName,
`timestamp'{]})); var propsTo = ee.List(\_params.imgPropsRename)
.cat(ee.List({[}\_params.datetimeName, `timestamp'{]})); var imgProps =
img.toDictionary(propsFrom).rename(\\
propsFrom, propsTo); // Subset points that intersect the given image.
var fcSub = fc.filterBounds(img.geometry()); // Reduce the image by
regions. return img.reduceRegions(\{\\
collection: fcSub,\\
reducer: \_params.reducer, scale: \_params.scale, crs: \_params.crs\\
\}) // Add metadata to each feature. .map(function(f) \{ return
f.set(imgProps);\\
\}); // Converts the feature collection of feature collections to a
single //feature collection. \}).flatten(); return results;\\
\}

\hypertarget{point-collection-creation}{%
\subsection{Point Collection Creation}\label{point-collection-creation}}

Below, we create a set of points that form the basis of the zonal
statistics calculations. Note that a unique plot\_id property is added
to each point. A unique plot or point ID is important to include in your
vector dataset for future filtering and joining.

var pts = ee.FeatureCollection({[}
ee.Feature(ee.Geometry.Point({[}-118.6010, 37.0777{]}), \{\\
plot\_id: 1 \}), ee.Feature(ee.Geometry.Point({[}-118.5896, 37.0778{]}),
\{\\
plot\_id: 2 \}), ee.Feature(ee.Geometry.Point({[}-118.5842, 37.0805{]}),
\{\\
plot\_id: 3 \}), ee.Feature(ee.Geometry.Point({[}-118.5994, 37.0936{]}),
\{\\
plot\_id: 4 \}), ee.Feature(ee.Geometry.Point({[}-118.5861, 37.0567{]}),
\{\\
plot\_id: 5 \})\\
{]});print(`Points of interest', pts);

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F52a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{neighborhood-statistic-examples}{%
\subsection{Neighborhood Statistic
Examples}\label{neighborhood-statistic-examples}}

The following examples demonstrate extracting raster neighborhood
statistics for the following:

\begin{itemize}
\tightlist
\item
  A single raster with elevation and slope bands
\item
  A multiband MODIS time series
\item
  A multiband Landsat time series
\end{itemize}

In each example, the points created in the previous section will be
buffered and then used as regions to extract zonal statistics for each
image in the image collection.

\hypertarget{topographic-variables}{%
\subsubsection{Topographic Variables}\label{topographic-variables}}

This example demonstrates how to calculate zonal statistics for a single
multiband image. This Digital Elevation Model (DEM) contains a single
topographic band representing elevation.

\#\#\#\#Buffer the Points

Nex, we will apply a 45 m radius buffer to the points defined previously
by mapping the bufferPoints function over the feature collection. The
radius is set to 45 m to correspond to the 90 m pixel resolution of the
DEM. In this case, circles are used instead of squares (set the second
argument as false, i.e., do not use bounds).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Buffer the points.  }
\KeywordTok{var}\NormalTok{ ptsTopo }\OperatorTok{=}\NormalTok{ pts}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\FunctionTok{bufferPoints}\NormalTok{(}\DecValTok{45}\OperatorTok{,} \KeywordTok{false}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\#\#\#\#Calculate Zonal Statistics

There are two important things to note about the zonalStats function
that this example addresses:

\begin{itemize}
\tightlist
\item
  It accepts only an ee.ImageCollection, not an ee.Image; single images
  must be wrapped in an ImageCollection.
\item
  It expects every image in the input image collection to have a
  timestamp property named `system:time\_start' with values representing
  milliseconds from 00:00:00 UTC on 1 January 1970. Most datasets should
  have this property, if not, one should be added.
\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Import the MERIT global elevation dataset.  }
\KeywordTok{var}\NormalTok{ elev }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}MERIT/DEM/v1\_0\_3\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Calculate slope from the DEM.  }
\KeywordTok{var}\NormalTok{ slope }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Terrain}\OperatorTok{.}\FunctionTok{slope}\NormalTok{(elev)}\OperatorTok{;}  
  
\CommentTok{// Concatenate elevation and slope as two bands of an image.  }
\KeywordTok{var}\NormalTok{ topo }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\FunctionTok{cat}\NormalTok{(elev}\OperatorTok{,}\NormalTok{ slope)  }
    \CommentTok{// Computed images do not have a \textquotesingle{}system:time\_start\textquotesingle{} property; add one based  }
    \CommentTok{// on when the data were collected.   .set(\textquotesingle{}system:time\_start\textquotesingle{}, ee.Date(\textquotesingle{}2000{-}01{-}01\textquotesingle{}).millis());  }
  
\CommentTok{// Wrap the single image in an ImageCollection for use in the  }
\CommentTok{// zonalStats function.  }
\KeywordTok{var}\NormalTok{ topoCol }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{([topo])}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Define arguments for the zonalStats function and then run it. Note that
we are accepting defaults for the reducer, scale, Coordinate Reference
System (CRS), and image properties to copy over to the resulting feature
collection. Refer to the function definition above for defaults.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define parameters for the zonalStats function.  }
\KeywordTok{var}\NormalTok{ params }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\DecValTok{0}\OperatorTok{,} \DecValTok{1}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{bandsRename}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}elevation\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}slope\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Extract zonal statistics per point per image.  }
\KeywordTok{var}\NormalTok{ ptsTopoStats }\OperatorTok{=} \FunctionTok{zonalStats}\NormalTok{(topoCol}\OperatorTok{,}\NormalTok{ ptsTopo}\OperatorTok{,}\NormalTok{ params)}\OperatorTok{;}\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Topo zonal stats table\textquotesingle{}}\OperatorTok{,}\NormalTok{ ptsTopoStats)}\OperatorTok{;}  
  
\CommentTok{// Display the layers on the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\OperatorTok{{-}}\FloatTok{118.5957}\OperatorTok{,} \FloatTok{37.0775}\OperatorTok{,} \DecValTok{13}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(topoCol}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{2400}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{4200}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Elevation\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(topoCol}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{60}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Slope\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(pts}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}purple\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Points\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(ptsTopo}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}yellow\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Points w/ buffer\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result is a copy of the buffered point feature collection with new
properties added for the region reduction of each selected image band
according to the given reducer. A part of the FeatureCollection is shown
in Fig. F5.2.1. The data in that FeatureCollection corresponds to a
table containing the information of Table F5.2.3. See Fig. F5.2.2 for a
graphical representation of the points and the topographic data being
summarized.

\begin{figure}

{\centering \includegraphics{./F5/image29.png}

}

\caption{Fig. F5.2.1 A part of the FeatureCollection produced by
calculating the zonal statistics}

\end{figure}

\begin{figure}

{\centering \includegraphics{./F5/image5.png}

}

\caption{Fig. F5.2.2 Sample points and topographic slope. Elevation and
slope values for regions intersecting each buffered point are reduced
and attached as properties of the points.}

\end{figure}

Table F5.2.3 Example output from zonalStats organized as a table. Rows
correspond to collection features and columns are feature properties.
Note that elevation and slope values in this table are rounded to the
nearest tenth for brevity.

plot\_id

timestamp

datetime

elevation

slope

1

946684800000

2000-01-01 00:00:00

2648.1

29.7

2

946684800000

2000-01-01 00:00:00

2888.2

33.9

3

946684800000

2000-01-01 00:00:00

3267.8

35.8

4

946684800000

2000-01-01 00:00:00

2790.7

25.1

5

946684800000

2000-01-01 00:00:00

2559.4

29.4

\hypertarget{modis-time-series}{%
\subsubsection{MODIS Time Series}\label{modis-time-series}}

A time series of MODIS eight-day surface reflectance composites
demonstrates how to calculate zonal statistics for a multiband
ImageCollection that requires no preprocessing, such as cloud masking or
computation. Note that there is no built-in function for performing
region reductions on ImageCollection objects. The zonalStats function
that we are using for reduction is mapping the reduceRegions function
over an ImageCollection.

\#\#\#\#Buffer the Points

In this example, suppose the point collection represents center points
for field plots that are 100 m x 100 m, and apply a 50 m radius buffer
to the points to match the size of the plot. Since we want zonal
statistics for square plots, set the second argument of the bufferPoints
function to true, so that the bounds of the buffered points are
returned.

var ptsModis = pts.map(bufferPoints(50, true));

\#\#\#\#Calculate Zonal Statistic

Import the MODIS 500 m global eight-day surface reflectance composite
collection and filter the collection to include data for July, August,
and September from 2015 through 2019.

var modisCol = ee.ImageCollection(`MODIS/006/MOD09A1')\\
.filterDate(`2015-01-01', `2020-01-01')\\
.filter(ee.Filter.calendarRange(183, 245, `DAY\_OF\_YEAR'));

Reduce each image in the collection by each plot according to the
following parameters. Note that this time the reducer is defined as the
neighborhood median (ee.Reducer.median) instead of the default mean, and
that scale, CRS, and properties for the datetime are explicitly defined.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define parameters for the zonalStats function.  }
\KeywordTok{var}\NormalTok{ params }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:} \DecValTok{500}\OperatorTok{,}  
   \DataTypeTok{crs}\OperatorTok{:} \StringTok{\textquotesingle{}EPSG:5070\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}sur\_refl\_b01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}sur\_refl\_b02\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}sur\_refl\_b06\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{bandsRename}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}modis\_red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}modis\_nir\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}modis\_swir\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{datetimeName}\OperatorTok{:} \StringTok{\textquotesingle{}date\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{datetimeFormat}\OperatorTok{:} \StringTok{\textquotesingle{}YYYY{-}MM{-}dd\textquotesingle{}}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Extract zonal statistics per point per image.  }
\KeywordTok{var}\NormalTok{ ptsModisStats }\OperatorTok{=} \FunctionTok{zonalStats}\NormalTok{(modisCol}\OperatorTok{,}\NormalTok{ ptsModis}\OperatorTok{,}\NormalTok{ params)}\OperatorTok{;}\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Limited MODIS zonal stats table\textquotesingle{}}\OperatorTok{,}\NormalTok{ ptsModisStats}\OperatorTok{.}\FunctionTok{limit}\NormalTok{(}\DecValTok{50}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result is a feature collection with a feature for all combinations
of plots and images. Interpreted as a table, the result has 200 rows (5
plots times 40 images) and as many columns as there are feature
properties. Feature properties include those from the plot asset and the
image, and any associated non-system image properties. Note that the
printed results are limited to the first 50 features for brevity.

\hypertarget{landsat-time-series}{%
\subsubsection{Landsat Time Series}\label{landsat-time-series}}

This example combines Landsat surface reflectance imagery across three
instruments: Thematic Mapper (TM) from Landsat 5, Enhanced Thematic
Mapper Plus (ETM+) from Landsat 7, and Operational Land Imager (OLI)
from Landsat 8.

The following section prepares these collections so that band names are
consistent and cloud masks are applied. Reflectance among corresponding
bands are roughly congruent for the three sensors when using the surface
reflectance product; therefore the processing steps that follow do not
address inter-sensor harmonization. Review the current literature on
inter-sensor harmonization practices if you'd like to apply a
correction.

\#\#\#\#Prepare the Landsat Image Collection

First, define the function to mask cloud and shadow pixels (See Chap.
F4.3 for more detail on cloud masking).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Mask clouds from images and apply scaling factors.  }
\KeywordTok{function} \FunctionTok{maskScale}\NormalTok{(img) \{   }\KeywordTok{var}\NormalTok{ qaMask }\OperatorTok{=}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}QA\_PIXEL\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{bitwiseAnd}\NormalTok{(}\PreprocessorTok{parseInt}\NormalTok{(}\StringTok{\textquotesingle{}11111\textquotesingle{}}\OperatorTok{,}       \DecValTok{2}\NormalTok{))}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ saturationMask }\OperatorTok{=}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}QA\_RADSAT\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Apply the scaling factors to the appropriate bands.   var getFactorImg = function(factorNames) \{       var factorList = img.toDictionary().select(factorNames)  }
           \OperatorTok{.}\FunctionTok{values}\NormalTok{()}\OperatorTok{;}       \ControlFlowTok{return}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\FunctionTok{constant}\NormalTok{(factorList)}\OperatorTok{;}  
\NormalTok{   \}}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ scaleImg }\OperatorTok{=} \FunctionTok{getFactorImg}\NormalTok{([}\StringTok{\textquotesingle{}REFLECTANCE\_MULT\_BAND\_.\textquotesingle{}}\NormalTok{])}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ offsetImg }\OperatorTok{=} \FunctionTok{getFactorImg}\NormalTok{([}\StringTok{\textquotesingle{}REFLECTANCE\_ADD\_BAND\_.\textquotesingle{}}\NormalTok{])}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ scaled }\OperatorTok{=}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SR\_B.\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(scaleImg)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(  }
\NormalTok{   offsetImg)}\OperatorTok{;}   \CommentTok{// Replace the original bands with the scaled ones and apply the masks.   return img.addBands(scaled, null, true)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(qaMask)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(saturationMask)}\OperatorTok{;}  
\NormalTok{\}}

\NormalTok{Next}\OperatorTok{,}\NormalTok{ define functions to select and rename the bands }\KeywordTok{of}\NormalTok{ interest }\ControlFlowTok{for}\NormalTok{ the Operational Land }\FunctionTok{Imager}\NormalTok{ (OLI) aboard Landsat }\DecValTok{8}\OperatorTok{,}\NormalTok{ and }\ControlFlowTok{for}\NormalTok{ the TM}\OperatorTok{/}\NormalTok{ETM}\OperatorTok{+}\NormalTok{ imagers aboard earlier Landsats}\OperatorTok{.} \AttributeTok{This}\NormalTok{ is important because the band numbers are different }\ControlFlowTok{for}\NormalTok{ OLI and TM}\OperatorTok{/}\NormalTok{ETM}\OperatorTok{+,}\NormalTok{ and it will make future index calculations easier}\OperatorTok{.}

\CommentTok{// Selects and renames bands of interest for Landsat OLI.  }
\KeywordTok{function} \FunctionTok{renameOli}\NormalTok{(img) \{   }\ControlFlowTok{return}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(  }
\NormalTok{       [}\StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B6\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
\NormalTok{       [}\StringTok{\textquotesingle{}Blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Green\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\NormalTok{\}  }
  
\CommentTok{// Selects and renames bands of interest for TM/ETM+.  }
\KeywordTok{function} \FunctionTok{renameEtm}\NormalTok{(img) \{   }\ControlFlowTok{return}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(  }
\NormalTok{       [}\StringTok{\textquotesingle{}SR\_B1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B5\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B7\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
\NormalTok{       [}\StringTok{\textquotesingle{}Blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Green\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
\NormalTok{\}}

\NormalTok{Combine the cloud mask and band renaming functions into preparation functions }\ControlFlowTok{for}\NormalTok{ OLI and TM}\OperatorTok{/}\NormalTok{ETM}\OperatorTok{+.} \AttributeTok{Add}\NormalTok{ any other sensor}\OperatorTok{{-}}\NormalTok{specific preprocessing steps that you’d like to the functions below}\OperatorTok{.}

\CommentTok{// Prepares (cloud masks and renames) OLI images.  }
\KeywordTok{function} \FunctionTok{prepOli}\NormalTok{(img) \{  }
\NormalTok{   img }\OperatorTok{=} \FunctionTok{maskScale}\NormalTok{(img)}\OperatorTok{;}  
\NormalTok{   img }\OperatorTok{=} \FunctionTok{renameOli}\NormalTok{(img)}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ img}\OperatorTok{;}  
\NormalTok{\}}\CommentTok{// Prepares (cloud masks and renames) TM/ETM+ images.  }
\KeywordTok{function} \FunctionTok{prepEtm}\NormalTok{(img) \{  }
\NormalTok{   img }\OperatorTok{=} \FunctionTok{maskScale}\NormalTok{(img)}\OperatorTok{;}  
\NormalTok{   img }\OperatorTok{=} \FunctionTok{renameEtm}\NormalTok{(img)}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ img}\OperatorTok{;}  
\NormalTok{\}}

\NormalTok{Get the Landsat surface reflectance collections }\ControlFlowTok{for}\NormalTok{ OLI}\OperatorTok{,}\NormalTok{ ETM}\OperatorTok{+,}\NormalTok{ and TM sensors}\OperatorTok{.} \AttributeTok{Filter}\NormalTok{ them by the bounds }\KeywordTok{of}\NormalTok{ the point feature collection and apply the relevant image preparation }\KeywordTok{function}\OperatorTok{.}

\KeywordTok{var}\NormalTok{ ptsLandsat }\OperatorTok{=}\NormalTok{ pts}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\FunctionTok{bufferPoints}\NormalTok{(}\DecValTok{15}\OperatorTok{,} \KeywordTok{true}\NormalTok{))}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ oliCol }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(ptsLandsat)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(prepOli)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ etmCol }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LE07/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(ptsLandsat)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(prepEtm)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ tmCol }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LT05/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(ptsLandsat)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(prepEtm)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Merge the prepared sensor collections.

var landsatCol = oliCol.merge(etmCol).merge(tmCol);

\#\#\#\#Calculate Zonal Statistics

Reduce each image in the collection by each plot according to the
following parameters. Note that this example defines the imgProps and
imgPropsRename parameters to copy over and rename just two selected
image properties: Landsat image ID and the satellite that collected the
data. It also uses the max reducer, which, as an unweighted reducer,
will return the maximum value from pixels that have their centroid
within the buffer (see Sect. 4.1 below for more details).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define parameters for the zonalStats function.  }
\KeywordTok{var}\NormalTok{ params }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{max}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:} \DecValTok{30}\OperatorTok{,}  
   \DataTypeTok{crs}\OperatorTok{:} \StringTok{\textquotesingle{}EPSG:5070\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}Blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Green\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}NIR\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SWIR2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{bandsRename}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}ls\_blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ls\_green\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ls\_red\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ls\_nir\textquotesingle{}}\OperatorTok{,}       \StringTok{\textquotesingle{}ls\_swir1\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ls\_swir2\textquotesingle{}}\NormalTok{   ]}\OperatorTok{,}  
   \DataTypeTok{imgProps}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SENSOR\_ID\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SPACECRAFT\_ID\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{imgPropsRename}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}img\_id\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}satellite\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{datetimeName}\OperatorTok{:} \StringTok{\textquotesingle{}date\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{datetimeFormat}\OperatorTok{:} \StringTok{\textquotesingle{}YYYY{-}MM{-}dd\textquotesingle{}}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Extract zonal statistics per point per image.  }
\KeywordTok{var}\NormalTok{ ptsLandsatStats }\OperatorTok{=} \FunctionTok{zonalStats}\NormalTok{(landsatCol}\OperatorTok{,}\NormalTok{ ptsLandsat}\OperatorTok{,}\NormalTok{ params)   }\CommentTok{// Filter out observations where image pixels were all masked.   .filter(ee.Filter.notNull(params.bandsRename));  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Limited Landsat zonal stats table\textquotesingle{}}\OperatorTok{,}\NormalTok{ ptsLandsatStats}\OperatorTok{.}\FunctionTok{limit}\NormalTok{(}\DecValTok{50}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result is a feature collection with a feature for all combinations
of plots and images.

\#\#\#\#Dealing with Large Collections

If your browser times out, try exporting the results (as described in
Chap. F6.2). It's likely that point feature collections that cover a
large area or contain many points (point-image observations) will need
to be exported as a batch task by either exporting the final feature
collection as an asset or as a CSV/shapefile/GeoJSON to Google Drive or
GCS.

Here is how you would export the above Landsat image-point feature
collection to an asset and to Google Drive. Run the following code,
activate the Code Editor Tasks tab, and then click the Run button. If
you don't specify your own existing folder in Drive, the folder
``EEFA\_outputs'' will be created.

Export.table.toAsset(\{\\
collection: ptsLandsatStats,\\
description: `EEFA\_export\_Landsat\_to\_points',\\
assetId: `EEFA\_export\_values\_to\_points'\\
\});

Export.table.toDrive(\{\\
collection: ptsLandsatStats,\\
folder: `EEFA\_outputs', // this will create a new folder if it doesn't
exist description: `EEFA\_export\_values\_to\_points',\\
fileFormat: `CSV'\\
\});

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F52b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{additional-notes}{%
\subsection{Additional Notes}\label{additional-notes}}

\hypertarget{weighted-versus-unweighted-region-reduction}{%
\subsubsection{Weighted Versus Unweighted Region
Reduction}\label{weighted-versus-unweighted-region-reduction}}

A region used for calculation of zonal statistics often bisects multiple
pixels. Should partial pixels be included in zonal statistics? Earth
Engine lets you decide by allowing you to define a reducer as either
weighted or unweighted (or you can provide per-pixel weight
specification as an image band). A weighted reducer will include partial
pixels in the zonal statistic calculation by weighting each pixel's
contribution according to the fraction of the area intersecting the
region. An unweighted reducer, on the other hand, gives equal weight to
all pixels whose cell center intersects the region; all other pixels are
excluded from calculation of the statistic.

For aggregate reducers like ee.Reducer.mean and ee.Reducer.median, the
default mode is weighted, while identifier reducers such as
ee.Reducer.min and ee.Reducer.max are unweighted. You can adjust the
behavior of weighted reducers by calling unweighted on them, as in
ee.Reducer.mean.unweighted. You may also specify the weights by
modifying the reducer with splitWeights; however, that is beyond the
scope of this book.

\hypertarget{copy-properties-to-computed-images}{%
\subsubsection{Copy Properties to Computed
Images}\label{copy-properties-to-computed-images}}

Derived, computed images do not retain the properties of their source
image, so be sure to copy properties to computed images if you want them
included in the region reduction table. For instance, consider the
simple computation of unscaling Landsat SR data:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define a Landsat image.  }
\KeywordTok{var}\NormalTok{ img }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Print its properties.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}All image properties\textquotesingle{}}\OperatorTok{,}\NormalTok{ img}\OperatorTok{.}\FunctionTok{propertyNames}\NormalTok{())}\OperatorTok{;}  
  
\CommentTok{// Subset the reflectance bands and unscale them.  }
\KeywordTok{var}\NormalTok{ computedImg }\OperatorTok{=}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SR\_B.\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\FloatTok{0.0000275}\NormalTok{)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(}\OperatorTok{{-}}\FloatTok{0.2}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Print the unscaled image\textquotesingle{}s properties.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Lost original image properties\textquotesingle{}}\OperatorTok{,}\NormalTok{ computedImg}\OperatorTok{.}\FunctionTok{propertyNames}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Notice how the computed image does not have the source image's
properties and only retains the bands information. To fix this, use the
copyProperties function to add desired source properties to the derived
image. It is best practice to copy only the properties you really need
because some properties, such as those containing geometry objects,
lists, or feature collections, can significantly increase the
computational burden for large collections.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Subset the reflectance bands and unscale them, keeping selected  }
\CommentTok{// source properties.  }
\KeywordTok{var}\NormalTok{ computedImg }\OperatorTok{=}\NormalTok{ img}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}SR\_B.\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\FloatTok{0.0000275}\NormalTok{)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(}\OperatorTok{{-}}\FloatTok{0.2}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{copyProperties}\NormalTok{(img}\OperatorTok{,}\NormalTok{ [}\StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}LANDSAT\_PRODUCT\_ID\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Print the unscaled image\textquotesingle{}s properties.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Selected image properties retained\textquotesingle{}}\OperatorTok{,}\NormalTok{ computedImg  }
\OperatorTok{.}\FunctionTok{propertyNames}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now selected properties are included. Use this technique when returning
computed, derived images in a mapped function, and in single-image
operations.

\hypertarget{understanding-which-pixels-are-included-in-polygon-statistics}{%
\subsubsection{Understanding Which Pixels are Included in Polygon
Statistics}\label{understanding-which-pixels-are-included-in-polygon-statistics}}

If you want to visualize what pixels are included in a polygon for a
region reducer, you can adapt the following code to use your own region
(by replacing geometry), dataset, desired scale, and CRS parameters. The
important part to note is that the image data you are adding to the map
is reprojected using the same scale and CRS as that used in your region
reduction (see Fig. F5.2.3).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define polygon geometry.  }
\KeywordTok{var}\NormalTok{ geometry }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{(  }
\NormalTok{   [  }
\NormalTok{       [  }
\NormalTok{           [}\OperatorTok{{-}}\FloatTok{118.6019835717645}\OperatorTok{,} \FloatTok{37.079867782687884}\NormalTok{]}\OperatorTok{,}  
\NormalTok{           [}\OperatorTok{{-}}\FloatTok{118.6019835717645}\OperatorTok{,} \FloatTok{37.07838698844939}\NormalTok{]}\OperatorTok{,}  
\NormalTok{           [}\OperatorTok{{-}}\FloatTok{118.60036351751951}\OperatorTok{,} \FloatTok{37.07838698844939}\NormalTok{]}\OperatorTok{,}  
\NormalTok{           [}\OperatorTok{{-}}\FloatTok{118.60036351751951}\OperatorTok{,} \FloatTok{37.079867782687884}\NormalTok{]  }
\NormalTok{       ]  }
\NormalTok{   ]}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{false}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Import the MERIT global elevation dataset.  }
\KeywordTok{var}\NormalTok{ elev }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\StringTok{\textquotesingle{}MERIT/DEM/v1\_0\_3\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Define desired scale and crs for region reduction (for image display too).  }
\KeywordTok{var}\NormalTok{ proj }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{scale}\OperatorTok{:} \DecValTok{90}\OperatorTok{,}  
   \DataTypeTok{crs}\OperatorTok{:} \StringTok{\textquotesingle{}EPSG:5070\textquotesingle{}}  
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The count reducer will return how many pixel centers are overlapped by
the polygon region, which would be the number of pixels included in any
unweighted reducer statistic. You can also visualize which pixels will
be included in the reduction by using the toCollection reducer on a
latitude/longitude image and adding resulting coordinates as feature
geometry. Be sure to specify CRS and scale for both the region reducers
and the reprojected layer added to the map (see bullet list below for
more details).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// A count reducer will return how many pixel centers are overlapped by the  }
\CommentTok{// polygon region.  }
\KeywordTok{var}\NormalTok{ count }\OperatorTok{=}\NormalTok{ elev}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{.}\FunctionTok{reduceRegion}\NormalTok{(\{  }
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{count}\NormalTok{()}\OperatorTok{,}  
   \DataTypeTok{geometry}\OperatorTok{:}\NormalTok{ geometry}\OperatorTok{,}  
   \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ proj}\OperatorTok{.}\AttributeTok{scale}\OperatorTok{,}   \DataTypeTok{crs}\OperatorTok{:}\NormalTok{ proj}\OperatorTok{.}\AttributeTok{crs}  
\NormalTok{\})}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}n pixels in the reduction\textquotesingle{}}\OperatorTok{,}\NormalTok{ count}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}dem\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Make a feature collection of pixel center points for those that are  }
\CommentTok{// included in the reduction.  }
\KeywordTok{var}\NormalTok{ pixels }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\FunctionTok{pixelLonLat}\NormalTok{()}\OperatorTok{.}\FunctionTok{reduceRegion}\NormalTok{(\{  }
   \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{toCollection}\NormalTok{([}\StringTok{\textquotesingle{}lon\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}lat\textquotesingle{}}\NormalTok{])}\OperatorTok{,}  
   \DataTypeTok{geometry}\OperatorTok{:}\NormalTok{ geometry}\OperatorTok{,}  
    \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ proj}\OperatorTok{.}\AttributeTok{scale}\OperatorTok{,}   \DataTypeTok{crs}\OperatorTok{:}\NormalTok{ proj}\OperatorTok{.}\AttributeTok{crs}  
\NormalTok{\})}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ pixelsFc }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(pixels}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}features\textquotesingle{}}\NormalTok{))}\OperatorTok{.}\FunctionTok{map}\NormalTok{(   }\KeywordTok{function}\NormalTok{(f) \{       }\ControlFlowTok{return}\NormalTok{ f}\OperatorTok{.}\FunctionTok{setGeometry}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([f}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}lon\textquotesingle{}}\NormalTok{)}\OperatorTok{,}\NormalTok{ f  }
           \OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}lat\textquotesingle{}}\NormalTok{)  }
\NormalTok{       ]))}\OperatorTok{;}  
\NormalTok{   \})}\OperatorTok{;}  
  
\CommentTok{// Display layers on the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(geometry}\OperatorTok{,} \DecValTok{18}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(  }
\NormalTok{   elev}\OperatorTok{.}\FunctionTok{reproject}\NormalTok{(\{  }
       \DataTypeTok{crs}\OperatorTok{:}\NormalTok{ proj}\OperatorTok{.}\AttributeTok{crs}\OperatorTok{,}  
       \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ proj}\OperatorTok{.}\AttributeTok{scale}\NormalTok{   \})}\OperatorTok{,}  
\NormalTok{   \{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{2500}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:} \DecValTok{3000}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]  }
\NormalTok{   \}}\OperatorTok{,} \StringTok{\textquotesingle{}Image\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(geometry}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}white\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Geometry\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(pixelsFc}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}purple\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Pixels in reduction\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F5/image44.png}

}

\caption{Fig. F5.2.3 Identifying pixels used in zonal statistics. By
mapping the image and vector together, you can see which pixels are
included in the unweighted statistic. For this example, three pixels
would be included in the statistic because the polygon covers the center
point of three pixels.}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F52c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Finally, here are some notes on CRS and scale:

\begin{itemize}
\tightlist
\item
  Earth Engine runs reduceRegion using the projection of the image's
  first band if the CRS is unspecified in the function. For imagery
  spanning multiple UTM zones, for example, this would lead to different
  origins. For some functions Earth Engine uses the default EPSG:4326.
  Therefore, when the opportunity is presented, such as by the
  reduceRegion function, it is important to specify the scale and CRS
  explicitly.
\item
  The Map default CRS is EPSG:3857. When looking closely at pixels on
  the map, the data layer scale and CRS should also be set explicitly.
  Note that zooming out after setting a relatively small scale when
  reprojecting may result in memory and/or timeout errors because
  optimized pyramid layers for each zoom level will not be used.
\item
  Specifying the CRS and scale in both the reduceRegion and addLayer
  functions allows the map visualization to align with the information
  printed in the Console.
\item
  The Earth Engine default, WGS 84 lat long (EPSG:4326), is a generic
  CRS that works worldwide. The code above reprojects to EPSG:5070,
  North American Equal Albers, which is a CRS that preserves area for
  North American locations. Use the CRS that is best for your use case
  when adapting this to your own project, or maintain (and specify) the
  CRS of the image using, for example, crs: `img.projection().crs()'.
\end{itemize}

\hypertarget{conclusion-19}{%
\subsection*{Conclusion}\label{conclusion-19}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you used functions containing optional parameters to
extract raster values for collocated points. You also learned how to
buffer points, and apply weighted and unweighted reducers to get
different types of zonal statistics. These functions were applied to
three examples that differed by raster dataset, reducer, spatial
resolution, and scale. Lastly, you covered related topics like weighting
of reducers and buffer visualization. Now you're ready to apply these
ideas to your own work!

\hypertarget{references-13}{%
\subsection*{References}\label{references-13}}
\addcontentsline{toc}{subsection}{References}

Cansler CA, McKenzie D (2012) How robust are burn severity indices when
applied in a new region? Evaluation of alternate field-based and
remote-sensing methods. Remote Sens 4:456--483.
https://doi.org/10.3390/rs4020456

Miller JD, Thode AE (2007) Quantifying burn severity in a heterogeneous
landscape with a relative version of the delta Normalized Burn Ratio
(dNBR). Remote Sens Environ 109:66--80.
https://doi.org/10.1016/j.rse.2006.12.006

\hypertarget{advanced-vector-operations}{%
\section{Advanced Vector Operations}\label{advanced-vector-operations}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-18}{%
\subsubsection*{Author}\label{author-18}}

Ujaval Gandhi

\hypertarget{overview-20}{%
\subsubsection*{Overview}\label{overview-20}}

This chapter covers advanced techniques for visualizing and analyzing
vector data in Earth Engine. There are many ways to visualize feature
collections, and you will learn how to pick the appropriate method to
create visualizations, such as a choropleth map. We will also cover
geoprocessing techniques involving multiple vector layers, such as
selecting features in one layer by their proximity to features in
another layer and performing spatial joins.

\hypertarget{learning-outcomes-20}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-20}}

\begin{itemize}
\tightlist
\item
  Visualizing any vector dataset and creating a thematic map.
\item
  Understanding joins in Earth Engine.
\item
  Carrying out geoprocessing tasks with vector layers in Earth Engine.
\end{itemize}

\hypertarget{assumes-you-know-how-to-20}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-20}}

\begin{itemize}
\tightlist
\item
  Filter a FeatureCollection to obtain a subset (Chap. F5.0, Chap.
  F5.1).
\item
  Write a function and map it over a FeatureCollection (Chap. F5.1,
  Chap. F5.2).
\end{itemize}

\end{tcolorbox}

\hypertarget{visualizing-feature-collections}{%
\subsection{Visualizing Feature
Collections}\label{visualizing-feature-collections}}

There is a distinct difference between how rasters and vectors are
visualized. While images are typically visualized based on pixel values,
vector layers use feature properties (i.e., attributes) to create a
visualization. Vector layers are rendered on the Map by assigning a
value to the red, green, and blue channels for each pixel on the screen
based on the geometry and attributes of the features. The functions used
for vector data visualization in Earth Engine are listed below in
increasing order of complexity.

\begin{itemize}
\tightlist
\item
  Map.addLayer: As with raster layers, you can add a FeatureCollection
  to the Map by specifying visualization parameters. This method
  supports only one visualization parameter: color. All features are
  rendered with the specified color.
\item
  draw: This function supports the parameters pointRadius and
  strokeWidth in addition to color. It renders all features of the layer
  with the specified parameters.
\item
  paint: This is a more powerful function that can render each feature
  with a different color and width based on the values in the specified
  property.
\item
  style: This is the most versatile function. It can apply a different
  style to each feature, including color, pointSize, pointShape, width,
  fillColor, and lineType.
\end{itemize}

In the exercises below, we will learn how to use each of these functions
and see how they can generate different types of maps.

\hypertarget{creating-a-choropleth-map}{%
\subsubsection{Creating a Choropleth
Map}\label{creating-a-choropleth-map}}

We will use the TIGER: US Census Blocks layer, which stores census block
boundaries and their characteristics within the United States, along
with the San Francisco neighborhoods layer from Chap. F5.0 to create a
population density map for the city of San Francisco.

We start by loading the census blocks and San Francisco neighborhoods
layers. We use ee.Filter.bounds to filter the census blocks layer to the
San Francisco boundary.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ blocks }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}TIGER/2010/Blocks\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ roads }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}TIGER/2016/Roads\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ sfNeighborhoods }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(   }\StringTok{\textquotesingle{}projects/gee{-}book/assets/F5{-}0/SFneighborhoods\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ geometry }\OperatorTok{=}\NormalTok{ sfNeighborhoods}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(geometry)}\OperatorTok{;}  
  
\CommentTok{// Filter blocks to the San Francisco boundary.  }
\KeywordTok{var}\NormalTok{ sfBlocks }\OperatorTok{=}\NormalTok{ blocks}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{bounds}\NormalTok{(geometry))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The simplest way to visualize this layer is to use Map.addLayer (Fig.
F5.3.1). We can specify a color value in the visParams parameter of the
function. Each census block polygon will be rendered with stroke and
fill of the specified color. The fill color is the same as the stroke
color but has a 66\% opacity.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Visualize with a single color.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfBlocks}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}\#de2d26\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Census Blocks (single color)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F5/image34.png}

}

\caption{Fig. F5.3.1 San Francisco census blocks}

\end{figure}

The census blocks table has a property named `pop10' containing the
population totals as of the 2010 census. We can use this to create a
choropleth map showing population density. We first need to compute the
population density for each feature and add it as a property. To add a
new property to each feature, we can map a function over the
FeatureCollection and calculate the new property called `pop\_density'.
Earth Engine provides the area function, which can calculate the area of
a feature in square meters. We convert it to square miles and calculate
the population density per square mile.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Add a pop\_density column.  }
\KeywordTok{var}\NormalTok{ sfBlocks }\OperatorTok{=}\NormalTok{ sfBlocks}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(f) \{   }\CommentTok{// Get the polygon area in square miles.   }
\KeywordTok{var}\NormalTok{ area\_sqmi }\OperatorTok{=}\NormalTok{ f}\OperatorTok{.}\FunctionTok{area}\NormalTok{()}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(}\FloatTok{2.59e6}\NormalTok{)}\OperatorTok{;}   
\KeywordTok{var}\NormalTok{ population }\OperatorTok{=}\NormalTok{ f}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}pop10\textquotesingle{}}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Calculate population density.   }
\KeywordTok{var}\NormalTok{ density }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(population)}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(area\_sqmi)}\OperatorTok{;}   
\ControlFlowTok{return}\NormalTok{ f}\OperatorTok{.}\FunctionTok{set}\NormalTok{(\{       }
    \StringTok{\textquotesingle{}area\_sqmi\textquotesingle{}}\OperatorTok{:}\NormalTok{ area\_sqmi}\OperatorTok{,}       
    \StringTok{\textquotesingle{}pop\_density\textquotesingle{}}\OperatorTok{:}\NormalTok{ density  }
\NormalTok{   \})}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now we can use the paint function to create an image from this
FeatureCollection using the pop\_density property. The paint function
needs an empty image that needs to be cast to the appropriate data type.
Let's use the aggregate\_stats function to calculate basic statistics
for the given column of a FeatureCollection.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Calculate the statistics of the newly computed column.  }
\KeywordTok{var}\NormalTok{ stats }\OperatorTok{=}\NormalTok{ sfBlocks}\OperatorTok{.}\FunctionTok{aggregate\_stats}\NormalTok{(}\StringTok{\textquotesingle{}pop\_density\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(stats)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

You will see that the population density values have a large range. We
also have values that are greater than 100,000, so we need to make sure
we select a data type that can store values of this size. We create an
empty image and cast it to int32, which is able to hold large integer
values.

The result is an image with pixel values representing the population
density of the polygons. We can now use the standard image visualization
method to add this layer to the Map (Fig. F5.3.2). Then, we need to
determine minimum and maximum values for the visualization parameters.A
reliable technique to produce a good visualization is to find minimum
and maximum values that are within one standard deviation. From the
statistics that we calculated earlier, we can estimate good minimum and
maximum values to be 0 and 50000, respectively.

var palette = {[}`fee5d9', `fcae91', `fb6a4a', `de2d26', `a50f15'{]};\\
var visParams = \{\\
min: 0,\\
max: 50000,\\
palette: palette\\
\};\\
Map.addLayer(sfBlocksPaint.clip(geometry), visParams, `Population
Density');

\begin{figure}

{\centering \includegraphics{./F5/image41.png}

}

\caption{Fig. F5.3.2 San Francisco population density}

\end{figure}

\hypertarget{creating-a-categorical-map}{%
\subsubsection{Creating a Categorical
Map}\label{creating-a-categorical-map}}

Continuing the exploration of styling methods, we will now learn about
draw and style. These are the preferred methods of styling for points
and line layers. Let's see how we can visualize the TIGER: US Census
Roads layer to create a categorical map.

We start by filtering the roads layer to the San Francisco boundary and
using Map.addLayer to visualize it.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Filter roads to San Francisco boundary.  }
\KeywordTok{var}\NormalTok{ sfRoads }\OperatorTok{=}\NormalTok{ roads}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{bounds}\NormalTok{(geometry))}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfRoads}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}blue\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Roads (default)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The default visualization renders each line using a width of 2 pixels.
The draw function provides a way to specify a different line width.
Let's use it to render the layer with the same color as before but with
a line width of 1 pixel (Fig. F5.3.3).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Visualize with draw().  }
\KeywordTok{var}\NormalTok{ sfRoadsDraw }\OperatorTok{=}\NormalTok{ sfRoads}\OperatorTok{.}\FunctionTok{draw}\NormalTok{(\{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}blue\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{strokeWidth}\OperatorTok{:} \DecValTok{1}  
\NormalTok{\})}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfRoadsDraw}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Roads (Draw)\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\includegraphics{./F5/image28.png}

\begin{figure}

{\centering \includegraphics{./F5/image31.png}

}

\caption{Fig. F5.3.3 San Francisco roads rendered with a line width of 2
pixels (left) and and a line width of 1 pixel (right)}

\end{figure}

The road layer has a column called ``MTFCC'' (standing for the MAF/TIGER
Feature Class Code). This contains the road priority codes, representing
the various types of roads, such as primary and secondary. We can use
this information to render each road segment according to its priority.
The draw function doesn't allow us to specify different styles for each
feature. Instead, we need to make use of the style function.

The column contains string values indicating different road types as
indicated in Table F5.3.1. This full list is available at the MAF/TIGER
Feature Class Code Definitions page on the US Census Bureau website.

Table F5.3.1 Census Bureau road priority codes

MTFCC

Feature Class

S1100

Primary Road

S1200

Secondary Road

S1400

Local Neighborhood Road, Rural Road, City Street

S1500

Vehicular Trail

S1630

Ramp

S1640

Service Drive

S1710

Walkway/Pedestrian Trail

S1720

Stairway

S1730

Alley

S1740

Private Road for service vehicles

S1750

Internal U.S. Census Bureau use

S1780

Parking Lot Road

S1820

Bike Path or Trail

S1830

Bridle Path

S2000

Road Median

Let's say we want to create a map with rules based on the MTFCC values
shown in Table F5.3.2.

Table F5.3.2 Styling Parameters for Road Priority Codes

MTFCC

Color

Line Width

S1100

Blue

3

S1200

Green

2

S1400

Orange

1

All Other Classes

Gray

1

Let's define a dictionary containing the styling information.

var styles = ee.Dictionary(\{ `S1100': \{ `color': `blue', `width': 3
\}, `S1200': \{ `color': `green', `width': 2 \}, `S1400': \{ `color':
`orange', `width': 1 \}\\
\});var defaultStyle = \{\\
color: `gray', `width': 1\\
\};

The style function needs a property in the FeatureCollection that
contains a dictionary with the style parameters. This allows you to
specify a different style for each feature. To create a new property, we
map a function over the FeatureCollection and assign an appropriate
style dictionary to a new property named `style'. Note the use of the
get function, which allows us to fetch the value for a key in the
dictionary. It also takes a default value in case the specified key does
not exist. We make use of this to assign different styles to the three
road classes specified in Table 5.3.2 and a default style to all others.

var sfRoads = sfRoads.map(function(f) \{ var classcode = f.get(`mtfcc');
var style = styles.get(classcode, defaultStyle); return f.set(`style',
style);\\
\});

Our collection is now ready to be styled. We call the style function to
specify the property that contains the dictionary of style parameters.
The output of the style function is an RGB image rendered from the
FeatureCollection (Fig. F5.3.4).

var sfRoadsStyle = sfRoads.style(\{\\
styleProperty: `style'\\
\});\\
Map.addLayer(sfRoadsStyle.clip(geometry), \{\}, `Roads (Style)');

\begin{figure}

{\centering \includegraphics{./F5/image46.png}

}

\caption{Fig. F5.3.4 San Francisco roads rendered according to road
priority}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F53a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Save your script for your own future use, as outlined in Chap. F1.0.
Then, refresh the Code Editor to begin with a new script for the next
section.

\hypertarget{joins-with-feature-collections}{%
\subsection{Joins with Feature
Collections}\label{joins-with-feature-collections}}

Earth Engine was designed as a platform for processing raster data, and
that is where it shines. Over the years, it has acquired advanced vector
data processing capabilities, and users are now able to carry out
complex geoprocessing tasks within Earth Engine. You can leverage the
distributed processing power of Earth Engine to process large vector
layers in parallel.

This section shows how you can do spatial queries and spatial joins
using multiple large feature collections. This requires the use of
joins. As described for Image Collections in Chap. F4.9, a join allows
you to match every item in a collection with items in another collection
based on certain conditions. While you can achieve similar results using
map and filter, joins perform better and give you more flexibility. We
need to define the following items to perform a join on two collections.

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Filter: A filter defines the condition used to select the features
  from the two collections. There is a suite of filters in the
  ee.Filters module that work on two collections, such as
  ee.Filter.equals and ee.Filter.withinDistance.
\item
  Join type: While the filter determines which features will be joined,
  the join type determines how they will be joined. There are many join
  types, including simple join, inner join, and save-all join.
\end{enumerate}

Joins are one of the harder skills to master, but doing so will help you
perform many complex analysis tasks within Earth Engine. We will go
through practical examples that will help you understand these concepts
and the workflow better.

\hypertarget{selecting-by-location}{%
\subsubsection{Selecting by Location}\label{selecting-by-location}}

In this section, we will learn how to select features from one layer
that are within a specified distance from features in another layer. We
will continue to work with the San Francisco census blocks and roads
datasets from the previous section. We will implement a join to select
all blocks in San Francisco that are within 1 km of an interstate
highway.

We start by loading the census blocks and roads collections and
filtering the roads layer to the San Francisco boundary.

var blocks = ee.FeatureCollection(`TIGER/2010/Blocks');\\
var roads = ee.FeatureCollection(`TIGER/2016/Roads');\\
var sfNeighborhoods = ee.FeatureCollection(
`projects/gee-book/assets/F5-0/SFneighborhoods');

var geometry = sfNeighborhoods.geometry();\\
Map.centerObject(geometry);

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Filter blocks and roads to San Francisco boundary.  }
\KeywordTok{var}\NormalTok{ sfBlocks }\OperatorTok{=}\NormalTok{ blocks}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{bounds}\NormalTok{(geometry))}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ sfRoads }\OperatorTok{=}\NormalTok{ roads}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{bounds}\NormalTok{(geometry))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

As we want to select all blocks within 1 km of an interstate highway, we
first filter the sfRoads collection to select all segments with the
rttyp property value of I.

var interstateRoads = sfRoads.filter(ee.Filter.eq(`rttyp', `I'));

We use the draw function to visualize the sfBlocks and interstateRoads
layers (Fig. F5.3.5).

var sfBlocksDrawn = sfBlocks.draw(\{\\
color: `gray',\\
strokeWidth: 1 \})\\
.clip(geometry);\\
Map.addLayer(sfBlocksDrawn, \{\}, `All Blocks');\\
var interstateRoadsDrawn = interstateRoads.draw(\{\\
color: `blue',\\
strokeWidth: 3 \})\\
.clip(geometry);\\
Map.addLayer(interstateRoadsDrawn, \{\}, `Interstate Roads');

\begin{figure}

{\centering \includegraphics{./F5/image2.png}

}

\caption{Fig. F5.3.5 San Francisco blocks and interstate highways}

\end{figure}

Let's define a join that will select all the features from the sfBlocks
layer that are within 1 km of any feature from the interstateRoads
layer. We start by defining a filter using the ee.Filter.withinDistance
filter. We want to compare the geometries of features in both layers, so
we use a special property called `.geo' to compare the collections. By
default, the filter will work with exact distances between the
geometries. If your analysis does not require a very precise tolerance
of spatial uncertainty, specifying a small non-zero maxError distance
value will help speed up the spatial operations. A larger tolerance also
helps when testing or debugging code so you can get the result quickly
instead of waiting longer for a more precise output.

var joinFilter = ee.Filter.withinDistance(\{\\
distance: 1000,\\
leftField: `.geo',\\
rightField: `.geo',\\
maxError: 10\\
\});

We will use a simple join as we just want features from the first
(primary) collection that match the features from the other (secondary)
collection.

var closeBlocks = ee.Join.simple().apply(\{\\
primary: sfBlocks,\\
secondary: interstateRoads,\\
condition: joinFilter\\
\});

We can visualize the results in a different color and verify that the
join worked as expected (Fig. F5.3.6).

var closeBlocksDrawn = closeBlocks.draw(\{\\
color: `orange',\\
strokeWidth: 1 \})\\
.clip(geometry);\\
Map.addLayer(closeBlocksDrawn, \{\}, `Blocks within 1km');

\begin{figure}

{\centering \includegraphics{./F5/image40.png}

}

\caption{Fig. F5.3.6 Selected blocks within 1 km of an interstate
highway}

\end{figure}

\hypertarget{spatial-joins}{%
\subsubsection{Spatial Joins}\label{spatial-joins}}

A spatial join allows you to query two collections based on the spatial
relationship. We will now implement a spatial join to count points in
polygons. We will work with a dataset of tree locations in San Francisco
and polygons of neighborhoods to produce a CSV file with the total
number of trees in each neighborhood.

The San Francisco Open Data Portal maintains a street tree map dataset
that has a list of street trees with their latitude and longitude. We
will also use the San Francisco neighborhood dataset from the same
portal. We downloaded, processed, and uploaded these layers as Earth
Engine assets for use in this exercise. We start by loading both layers
and using the paint and style functions, covered in Sect. 1, to
visualize them (Fig. F5.3.7).

var sfNeighborhoods = ee.FeatureCollection(
`projects/gee-book/assets/F5-0/SFneighborhoods');\\
var sfTrees = ee.FeatureCollection(
`projects/gee-book/assets/F5-3/SFTrees');

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Use paint() to visualize the polygons with only outline  }
\KeywordTok{var}\NormalTok{ sfNeighborhoodsOutline }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{  }
   \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ sfNeighborhoods}\OperatorTok{,}  
   \DataTypeTok{color}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{width}\OperatorTok{:} \DecValTok{3}  
\NormalTok{\})}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfNeighborhoodsOutline}\OperatorTok{,}\NormalTok{ \{  }
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}blue\textquotesingle{}}\NormalTok{]  }
\NormalTok{   \}}\OperatorTok{,}   \StringTok{\textquotesingle{}SF Neighborhoods\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Use style() to visualize the points  }
\KeywordTok{var}\NormalTok{ sfTreesStyled }\OperatorTok{=}\NormalTok{ sfTrees}\OperatorTok{.}\FunctionTok{style}\NormalTok{(\{  }
   \DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}green\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{pointSize}\OperatorTok{:} \DecValTok{2}\OperatorTok{,}  
   \DataTypeTok{pointShape}\OperatorTok{:} \StringTok{\textquotesingle{}triangle\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{width}\OperatorTok{:} \DecValTok{2}  
\NormalTok{\})}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(sfTreesStyled}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}SF Trees\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F5/image35.png}

}

\caption{Fig. F5.3.7 San Francisco neighborhoods and trees}

\end{figure}

To find the tree points in each neighborhood polygon, we will use an
ee.Filter.intersects filter.

var intersectFilter = ee.Filter.intersects(\{\\
leftField: `.geo',\\
rightField: `.geo',\\
maxError: 10\\
\});

We need a join that can give us a list of all tree features that
intersect each neighborhood polygon, so we need to use a saving join. A
saving join will find all the features from the secondary collection
that match the filter and store them in a property in the primary
collection. Once you apply this join, you will get a version of the
primary collection with an additional property that has the matching
features from the secondary collection. Here we use the ee.Join.saveAll
join, since we want to store all matching features. We specify the
matchesKey property that will be added to each feature with the results.

var saveAllJoin = ee.Join.saveAll(\{\\
matchesKey: `trees',\\
\});

Let's apply the join and print the first feature of the resulting
collection to verify (Fig. F5.3.8).

var joined = saveAllJoin\\
.apply(sfNeighborhoods, sfTrees, intersectFilter);\\
print(joined.first());

\begin{figure}

{\centering \includegraphics{./F5/image1.png}

}

\caption{Fig. F5.3.8 Result of the save-all join}

\end{figure}

You will see that each feature of the sfNeighborhoods collection now has
an additional property called trees. This contains all the features from
the sfTrees collection that were matched using the intersectFilter. We
can now map a function over the results and post-process the collection.
As our analysis requires the computation of the total number of trees in
each neighborhood, we extract the matching features and use the size
function to get the count (Fig. F5.3.9).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Calculate total number of trees within each feature.  }
\KeywordTok{var}\NormalTok{ sfNeighborhoods }\OperatorTok{=}\NormalTok{ joined}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(f) \{   }\KeywordTok{var}\NormalTok{ treesWithin }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{List}\NormalTok{(f}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}trees\textquotesingle{}}\NormalTok{))}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ totalTrees }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(treesWithin)}\OperatorTok{.}\FunctionTok{size}\NormalTok{()}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ f}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{\textquotesingle{}total\_trees\textquotesingle{}}\OperatorTok{,}\NormalTok{ totalTrees)}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(sfNeighborhoods}\OperatorTok{.}\FunctionTok{first}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F5/image18.png}

}

\caption{Fig. F5.3.9 Final FeatureCollection with the new property}

\end{figure}

The results now have a property called total\_trees containing the count
of intersecting trees in each neighborhood polygon.

The final step in the analysis is to export the results as a CSV file
using the Export.table.toDrive function. Note that as described in
detail in F6.2, you should output only the columns you need to the CSV
file. Suppose we do not need all the properties to appear in the output;
imagine that wedo not need the trees property, for example, in the
output. In that case, we can create only those columns we want in the
manner below, by specifying the other selectors parameters with the list
of properties to export.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Export the results as a CSV.  }
\NormalTok{Export}\OperatorTok{.}\AttributeTok{table}\OperatorTok{.}\FunctionTok{toDrive}\NormalTok{(\{  }
   \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ sfNeighborhoods}\OperatorTok{,}  
   \DataTypeTok{description}\OperatorTok{:} \StringTok{\textquotesingle{}SF\_Neighborhood\_Tree\_Count\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{folder}\OperatorTok{:} \StringTok{\textquotesingle{}earthengine\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{fileNamePrefix}\OperatorTok{:} \StringTok{\textquotesingle{}tree\_count\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{fileFormat}\OperatorTok{:} \StringTok{\textquotesingle{}CSV\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{selectors}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}nhood\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}total\_trees\textquotesingle{}}\NormalTok{]  }
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The final result is a CSV file with the neighborhood names and total
numbers of trees counted using the join (Fig. F5.3.10).

\begin{figure}

{\centering \includegraphics{./F5/image3.png}

}

\caption{Fig. F5.3.10 Exported CSV file with tree counts for San
Francisco neighborhoods}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F53b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-20}{%
\subsection*{Conclusion}\label{conclusion-20}}
\addcontentsline{toc}{subsection}{Conclusion}

This chapter covered visualization and analysis using vector data in
Earth Engine. You should now understand different functions for
FeatureCollection visualization and be able to create thematic maps with
vector layers. You also learned techniques for doing spatial queries and
spatial joins within Earth Engine. Earth Engine is capable of handling
large feature collections and can be effectively used for many spatial
analysis tasks.

\hypertarget{advanced-topics}{%
\chapter{Advanced Topics}\label{advanced-topics}}

Although you now know the most basic fundamentals of Earth Engine, there
is still much more that can be done. The Part presents some advanced
topics that can help expand your skill set for doing larger and more
complex projects. These include tools for sharing code among users,
scaling up with efficient project design, creating apps for non-expert
users, and combining R with other information processing platforms.

\hypertarget{advanced-raster-visualization}{%
\section{Advanced Raster
Visualization}\label{advanced-raster-visualization}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-19}{%
\subsubsection*{Author}\label{author-19}}

Gennadii Donchyts, Fedor Baart

\hypertarget{overview-21}{%
\subsubsection*{Overview}\label{overview-21}}

This chapter should help users of Earth Engine to better understand
raster data by applying visualization algorithms such as hillshading,
hill shadows, and custom colormaps. We will also learn how image
collection datasets can be explored by animating them as well as by
annotating with text labels, using, for example, attributes of images or
values queried from images.

\hypertarget{learning-outcomes-21}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-21}}

\begin{itemize}
\tightlist
\item
  Understanding why perceptually uniform colormaps are better to present
  data and using them efficiently for raster visualization.
\item
  Using palettes with images before and after remapping values.
\item
  Adding text annotations when visualizing images or features.
\item
  Animating image collections in multiple ways (animated GIFs, exporting
  video clips, interactive animations with UI controls).
\item
  Adding hillshading and shadows to help visualize raster datasets.
\end{itemize}

\hypertarget{assumes-you-know-how-to-21}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-21}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Inspect an Image and an ImageCollection, as well as their properties
  (Chap. F4.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-17}{%
\subsection*{Introduction}\label{introduction-17}}

Visualization is the step to transform data into a visual
representation. You make a visualization as soon as you add your first
layer to your map in Google Earth Engine. Sometimes you just want to
have a first look at a dataset during the exploration phase. But as you
move towards the dissemination phase, where you want to spread your
results, it is good to think about a more structured approach to
visualization. A typical workflow for creating visualization consists of
the following steps:

\begin{itemize}
\tightlist
\item
  Defining the story (what is the message?)
\item
  Finding inspiration (for example by making a moodboard)
\item
  Choosing a canvas/medium (here, this is the Earth Engine map canvas)
\item
  Choosing datasets (co-visualized or combined using derived indicators)
\item
  Data preparation (interpolating in time and space,
  filtering/mapping/reducing)
\item
  Converting data into visual elements (shape and color)
\item
  Adding annotations and interactivity (labels, scales, legend, zoom,
  time slider)
\end{itemize}

A good standard work on all the choices that one can make while creating
a visualization is provided by the Grammar of Graphics (GoG) by
Wilkinson (1999). It was the inspiration behind many modern
visualization libraries (ggplot, vega). The main concept is that you can
subdivide your visualization into several aspects.

In this chapter, we will cover several aspects mentioned in the Grammar
of Graphics to convert (raster) data into visual elements. The accurate
representation of data is essential in science communication. However,
color maps that visually distort data through uneven color gradients or
are unreadable to those with color-vision deficiency remain prevalent in
science (Crameri, 2020). You will also learn how to add annotation text
and symbology, while improving your visualizations by mixing images with
hillshading as you explore some of the amazing datasets that have been
collected in recent years in Earth Engine.

\hypertarget{palettes}{%
\subsection{Palettes}\label{palettes}}

In this section we will explore examples of colormaps to visualize
raster data. Colormaps translate values to colors for display on a map.
This requires a set of colors (referred to as a ``palette'' in Earth
Engine) and a range of values to map (specified by the min and max
values in the visualization parameters).

There are multiple types of colormaps, each used for a different
purpose. These include the following:

Sequential: These are probably the most commonly used colormaps, and are
useful for ordinal, interval, and ratio data. Also referred to as a
linear colormap, a sequential colormap looks like the viridis colormap
(Fig. F6.0.1) from matplotlib. It is popular because it is a perceptual
uniform colormap, where an equal interval in values is mapped to an
equal interval in the perceptual colorspace. If you have a ratio
variable where zero means nothing, you can use a sequential colormap
starting at white, transparent, or, when you have a black background, at
black---for example, the turku colormap from Crameri (Fig. F6.0.1). You
can use this for variables like population count or gross domestic
product.

Diverging: This type of colormap is used for visualizing data where you
have positive and negative values and where zero has a meaning. Later in
this tutorial, we will use the balance colormap from the cmocean package
(Fig. F6.0.1) to show temperature change.

Circular: Some variables are periodic, returning to the same value after
a period of time. For example, the season, angle, and time of day are
typically represented as circular variables. For variables like this, a
circular colormap is designed to represent the first and last values
with the same color. An example is the circular cet-c2 colormap (Fig.
F6.0.1) from the colorcet package.

Semantic: Some colormaps do not map to arbitrary colors but choose
colors that provide meaning. We refer to these as semantic colormaps.
Later in this tutorial, we will use the ice colormap (Fig. F6.0.1) from
the cmocean package for our ice example.

\begin{figure}

{\centering \includegraphics{./F6/image40.png}

}

\caption{Fig. F6.0.1 Examples of colormaps from a variety of packages:
viridis from matplotlib, turku from Crameri, balance from cmocean,
cet-c2 from colorcet and ice from cmocean}

\end{figure}

Popular sources of colormaps include:

\begin{itemize}
\tightlist
\item
  cmocean (semantic perceptual uniform colormaps for geophysical
  applications)
\item
  colorcet (set of perceptual colormaps with varying colors and
  saturation)
\item
  cpt-city (comprehensive overview of colormaps,
\item
  colorbrewer (colormaps with variety of colors)
\item
  Crameri (stylish colormaps for dark and light themes)
\end{itemize}

Our first example in this section applies a diverging colormap to
temperature.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load the ERA5 reanalysis monthly means.  }
\KeywordTok{var}\NormalTok{ era5 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}ECMWF/ERA5\_LAND/MONTHLY\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Load the palettes package.  }
\KeywordTok{var}\NormalTok{ palettes }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:palettes\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Select temperature near ground.  }
\NormalTok{era5 }\OperatorTok{=}\NormalTok{ era5}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}temperature\_2m\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now we can visualize the data. Here we have a temperature difference.
That means that zero has a special meaning. By using a divergent
colormap we can give zero the color white, which denotes that there is
no significant difference. Here we will use the colormap Balance from
the cmocean package. The color red is associated with warmth, and the
color blue is associated with cold. We will choose the minimum and
maximum values for the palette to be symmetric around zero (-2, 2) so
that white appears in the correct place. For comparison we also
visualize the data with a simple {[}`blue', `white', `red'{]} palette.
As you can see (Fig. F6.0.2), the Balance colormap has a more elegant
and professional feel to it, because it uses a perceptual uniform
palette and both saturation and value.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Choose a diverging colormap for anomalies.  }
\KeywordTok{var}\NormalTok{ balancePalette }\OperatorTok{=}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{cmocean}\OperatorTok{.}\AttributeTok{Balance}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ threeColorPalette }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}blue\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Show the palette in the Inspector window.  }
\NormalTok{palettes}\OperatorTok{.}\FunctionTok{showPalette}\NormalTok{(}\StringTok{\textquotesingle{}temperature anomaly\textquotesingle{}}\OperatorTok{,}\NormalTok{ balancePalette)}\OperatorTok{;}  
\NormalTok{palettes}\OperatorTok{.}\FunctionTok{showPalette}\NormalTok{(}\StringTok{\textquotesingle{}temperature anomaly\textquotesingle{}}\OperatorTok{,}\NormalTok{ threeColorPalette)}\OperatorTok{;}  
  
\CommentTok{// Select 2 time windows of 10 years.  }
\KeywordTok{var}\NormalTok{ era5\_1980 }\OperatorTok{=}\NormalTok{ era5}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}1981{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1991{-}01{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ era5\_2010 }\OperatorTok{=}\NormalTok{ era5}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2011{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Compute the temperature change.  }
\KeywordTok{var}\NormalTok{ era5\_diff }\OperatorTok{=}\NormalTok{ era5\_2010}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(era5\_1980)}\OperatorTok{;}  
  
\CommentTok{// Show it on the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(era5\_diff}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ threeColorPalette}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{2}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{2}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Blue White Red palette\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(era5\_diff}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ balancePalette}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{2}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{2}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Balance palette\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\includegraphics{./F6/image66.png}\includegraphics{./F6/image53.png}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Our second example in this section focuses on visualizing a region of
the Antarctic, the Thwaites Glacier. This is one of the fast-flowing
glaciers that causes concern because it loses so much mass that it
causes the sea level to rise. If we want to visualize this region, we
have a challenge. The Antarctic region is in the dark for four to five
months each winter. That means that we can't use optical images to see
the ice flowing into the sea. We therefore will use radar images. Here
we will use a semantic colormap to denote the meaning of the radar
images.

Let's start by importing the dataset of radar images. We will use the
images from the Sentinel-1 constellation of the Copernicus program. This
satellite uses a C-band synthetic-aperture radar and has near-polar
coverage. The radar senses images using a polarity for the sender and
receiver. The collection has images of four different possible
combinations of sender/receiver polarity pairs. The image that we'll use
has a band of the Horizontal/Horizontal polarity (HH).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// An image of the Thwaites glacier.  }
\KeywordTok{var}\NormalTok{ imageId }\OperatorTok{=}\StringTok{\textquotesingle{}COPERNICUS/S1\_GRD/S1B\_EW\_GRDM\_1SSH\_20211216T041925\_20211216T042029\_030045\_03965B\_AF0A\textquotesingle{}}\OperatorTok{;}  
  
\CommentTok{// Look it up and select the HH band.  }
\KeywordTok{var}\NormalTok{ img }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(imageId)}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}HH\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

For the next step, we will use the palette library. We will stylize the
radar images to look like optical images, so that viewers can contrast
ice and sea ice from water (Lhermitte, 2020). We will use the Ice
colormap from the cmocean package (Thyng, 2016).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Use the palette library.  }
\KeywordTok{var}\NormalTok{ palettes }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:palettes\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Access the ice palette.  }
\KeywordTok{var}\NormalTok{ icePalette }\OperatorTok{=}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{cmocean}\OperatorTok{.}\AttributeTok{Ice}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Show it in the console.  }
\NormalTok{palettes}\OperatorTok{.}\FunctionTok{showPalette}\NormalTok{(}\StringTok{\textquotesingle{}Ice\textquotesingle{}}\OperatorTok{,}\NormalTok{ icePalette)}\OperatorTok{;}  
  
\CommentTok{// Use  it to visualize the radar data.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(img}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ icePalette}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{15}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Sentinel{-}1 radar\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Zoom to the grounding line of the Thwaites Glacier.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{105.45882094907664}\OperatorTok{,} \OperatorTok{{-}}   \FloatTok{74.90419580705336}\NormalTok{])}\OperatorTok{,} \DecValTok{8}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If you zoom in (F6.0.3) you can see how long cracks have recently
appeared near the pinning point (a peak in the bathymetry that functions
as a buttress, see Wild, 2022) of the glacier.

\begin{figure}

{\centering \includegraphics{./F6/image13.png}

}

\caption{Fig. F6.0.3. Ice observed in Antarctica by the Sentinel-1
satellite. The image is rendered using the ice color palette stretched
to backscatter amplitude values {[}-15; 1{]}.}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60b. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{remapping-and-palettes}{%
\subsection{Remapping and Palettes}\label{remapping-and-palettes}}

Classified rasters in Earth Engine have metadata attached that can help
with analysis and visualization. This includes lists of the names,
values, and colors associated with class. These are used as the default
color palette for drawing a classification, as seen next. The USGS
National Land Cover Database (NLCD) is one such example. Let's access
the NLCD dataset, name it nlcd, and view it (Fig. F6.0.4) with its
built-in palette.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Advanced remapping using NLCD.  }
\CommentTok{// Import NLCD.  }
\KeywordTok{var}\NormalTok{ nlcd }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}USGS/NLCD\_RELEASES/2016\_REL\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Use Filter to select the 2016 dataset.  }
\KeywordTok{var}\NormalTok{ nlcd2016 }\OperatorTok{=}\NormalTok{ nlcd}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{\textquotesingle{}system:index\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2016\textquotesingle{}}\NormalTok{))  }
   \OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Select the land cover band.  }
\KeywordTok{var}\NormalTok{ landcover }\OperatorTok{=}\NormalTok{ nlcd2016}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}landcover\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Map the NLCD land cover.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landcover}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \StringTok{\textquotesingle{}NLCD Landcover\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F6/image54.png}

}

\caption{Fig. F6.0.4 The NLCD visualized with default colors for each
class}

\end{figure}

But suppose you want to change the display palette. For example, you
might want to have multiple classes displayed using the same color, or
use different colors for some classes. Let's try having all three urban
classes display as dark red (`ab0000').

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Now suppose we want to change the color palette.  }
\KeywordTok{var}\NormalTok{ newPalette }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}466b9f\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}d1def8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}dec5c5\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}ab0000\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ab0000\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ab0000\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}b3ac9f\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}68ab5f\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}1c5f2c\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}b5c58f\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}af963c\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ccb879\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}dfdfc2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}d1d182\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}a3cc51\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}82ba9e\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}dcd939\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}ab6c28\textquotesingle{}}\OperatorTok{,}   \StringTok{\textquotesingle{}b8d9eb\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}6c9fb8\textquotesingle{}}  
\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Try mapping with the new color palette.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(landcover}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ newPalette  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NLCD New Palette\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

However, if you map this, you will see an unexpected result (Fig.
F6.0.5).

\begin{figure}

{\centering \includegraphics{./F6/image64.png}

}

\caption{Fig. F6.0.5 Applying a new palette to a multi-class layer has
some unexpected results}

\end{figure}

This is because the numeric codes for the different classes are not
sequential. Thus, Earth Engine stretches the given palette across the
whole range of values and produces an unexpected color palette. To fix
this issue, we will create a new index for the class values so that they
are sequential.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Extract the class values and save them as a list.  }
\KeywordTok{var}\NormalTok{ values }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{List}\NormalTok{(landcover}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}landcover\_class\_values\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Print the class values to console.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}raw class values\textquotesingle{}}\OperatorTok{,}\NormalTok{ values)}\OperatorTok{;}  
  
\CommentTok{// Determine the maximum index value  }
\KeywordTok{var}\NormalTok{ maxIndex }\OperatorTok{=}\NormalTok{ values}\OperatorTok{.}\FunctionTok{size}\NormalTok{()}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Create a new index for the remap  }
\KeywordTok{var}\NormalTok{ indexes }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{List}\OperatorTok{.}\FunctionTok{sequence}\NormalTok{(}\DecValTok{0}\OperatorTok{,}\NormalTok{ maxIndex)}\OperatorTok{;}  
  
\CommentTok{// Print the updated class values to console.  }
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}updated class values\textquotesingle{}}\OperatorTok{,}\NormalTok{ indexes)}\OperatorTok{;}  
  
\CommentTok{// Remap NLCD and display it in the map.  }
\KeywordTok{var}\NormalTok{ colorized }\OperatorTok{=}\NormalTok{ landcover}\OperatorTok{.}\FunctionTok{remap}\NormalTok{(values}\OperatorTok{,}\NormalTok{ indexes)  }
   \OperatorTok{.}\FunctionTok{visualize}\NormalTok{(\{  }
       \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
       \DataTypeTok{max}\OperatorTok{:}\NormalTok{ maxIndex}\OperatorTok{,}  
       \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ newPalette  }
\NormalTok{   \})}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(colorized}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}NLCD Remapped Colors\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Using this remapping approach, we can properly visualize the new color
palette (Fig. F6.0.6).

\begin{figure}

{\centering \includegraphics{./F6/image57.png}

}

\caption{Fig. F6.0.6 Expected results of the new color palette. All
urban areas are now correctly showing as dark red and the other land
cover types remain their original color.}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{annotations}{%
\subsection{Annotations}\label{annotations}}

Annotations are the way to visualize data on maps to provide additional
information about raster values or any other data relevant to the
context. In this case, this additional information is usually shown as
geometries, text labels, diagrams, or other visual elements. Some
annotations in Earth Engine can be added by making use of the ui portion
of the Earth Engine API, resulting in graphical user interface elements
such as labels or charts added on top of the map. However, it is
frequently useful to render annotations as a part of images, such as by
visualizing various image properties or to highlight specific areas.

In many cases, these annotations can be mixed with output images
generated outside of Earth Engine, for example, by post-processing
exported images using Python libraries or by annotating using GIS
applications such as QGIS or ArcGIS. However, annotations could also be
also very useful to highlight and/or label specific areas directly
within the Code Editor. Earth Engine provides a sufficiently rich API to
turn vector features and geometries into raster images which can serve
as annotations. We recommend checking the ee.FeatureCollection.style
function in the Earth Engine documentation to learn how geometries can
be rendered.

For textual annotation, we will make use of an external package
`users/gena/packages:text' that provides a way to render strings into
raster images directly using the Earth Engine raster API. It is beyond
the scope of the current tutorials to explain the implementation of this
package, but internally this package makes use of bitmap fonts which are
ingested into Earth Engine as raster assets and are used to turn every
character of a provided string into image glyphs, which are then
translated to desired coordinates.

The API of the text package includes the following mandatory and
optional arguments:

/**\\
* Draws a string as a raster image at a given point.\\
*\\
* (\textbf{param?}) \{string\} str - string to draw\\
* (\textbf{param?}) \{ee.Geometry\} point - location the the string will
be drawn\\
* (\textbf{param?}) \{\{string, Object\}\} options - optional properties
used to style text\\
*\\
* The options dictionary may include one or more of the following:\\
* fontSize - 16\textbar18\textbar24\textbar32 - the size of the font
(default: 16)\\
* fontType - Arial\textbar Consolas - the type of the font (default:
Arial)\\
* alignX - left\textbar center\textbar right (default: left)\\
* alignY - top\textbar center\textbar bottom (default: top)\\
* textColor - text color string (default: ffffff - white)\\
* textOpacity - 0-1, opacity of the text (default: 0.9)\\
* textWidth - width of the text (default: 1)\\
* outlineColor - text outline color string (default: 000000 - black)\\
* outlineOpacity - 0-1, opacity of the text outline (default: 0.4)\\
* outlineWidth - width of the text outlines (default: 0)\\
*/

To demonstrate how to use this API, let's render a simple `Hello World!'
text string placed at the map center using default text parameters. The
code for this will be:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Include the text package.  }
\KeywordTok{var}\NormalTok{ text }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:text\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Configure map (change center and map type).  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\DecValTok{0}\OperatorTok{,} \DecValTok{0}\OperatorTok{,} \DecValTok{10}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{\textquotesingle{}HYBRID\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Draw text string and add to map.  }
\KeywordTok{var}\NormalTok{ pt }\OperatorTok{=} \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getCenter}\NormalTok{()}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ scale }\OperatorTok{=} \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getScale}\NormalTok{()}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ image }\OperatorTok{=}\NormalTok{ text}\OperatorTok{.}\FunctionTok{draw}\NormalTok{(}\StringTok{\textquotesingle{}Hello World!\textquotesingle{}}\OperatorTok{,}\NormalTok{ pt}\OperatorTok{,}\NormalTok{ scale)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(image)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Running the above script will generate a new image containing the `Hello
World!' string placed in the map center. Notice that before calling the
text.draw() function we configure the map to be centered at specific
coordinates (0,0) and zoom level 10 because map parameters such as
center and scale are passed as arguments to that text.draw() function.
This ensures that the resulting image containing string characters is
scaled properly.

When exporting images containing rendered text strings, it is important
to use proper scale to avoid distorted text strings that are difficult
to read, depending on the selected font size, as shown in Fig. 6.0.7.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60d. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\includegraphics{./F6/image39.png}\includegraphics{./F6/image74.png}\includegraphics{./F6/image44.png}

These artifacts can be avoided to some extent by specifying a larger
font size (e.g., 32). However, it is better to render text at the native
1:1 scale to achieve best results. The same applies to the text color
and outline: They may need to be adjusted to achieve the best result.
Usually, text needs to be rendered using colors that have opposite
brightness and colors when compared to the surrounding background.
Notice that in the above example, the map was configured to have a dark
background (`HYBRID') to ensure that the white text (default color)
would be visible. Multiple parameters listed in the above API
documentation can be used to adjust text rendering. For example, let's
switch font size, font type, text, and outline parameters to render the
same string, as below. Replace the existing one-line text.draw call in
your script with the following code, and then run it again to see the
difference (Fig. F6.0.8):

var image = text.draw(`Hello World!', pt, scale, \{\\
fontSize: 32,\\
fontType: `Consolas',\\
textColor: `black',\\
outlineColor: `white',\\
outlineWidth: 1,\\
outlineOpacity: 0.8\\
\});

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Add the text image to the map.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(image)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60e. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\begin{figure}

{\centering \includegraphics{./F6/image33.png}

}

\caption{Fig. 6.0.8 Rendering text with adjusted parameters (font type:
Consolas, fontSize: 32, textColor: `black', outlineWidth: 1,
outlineColor: `white', outlineOpacity: 0.8)}

\end{figure}

Of course, non-optional parameters such as pt and scale, as well as the
text string, do not have to be hard-coded in the script; instead, they
can be acquired by the code using, for example, properties coming from a
FeatureCollection. Let's demonstrate this by showing the cloudiness of
Landsat 8 images as text labels rendered in the center of every image.
In addition to annotating every image with a cloudiness text string, we
will also draw yellow outlines to indicate image boundaries. For
convenience, we can also define the code to annotate an image as a
function. We will then map that function (as described in Chap. F4.0)
over the filtered ImageCollection. The code is as follows:

var text = require(`users/gena/packages:text');

var geometry = ee.Geometry.Polygon(\\
{[}\\
{[}\\
{[}-109.248, 43.3913{]},\\
{[}-109.248, 33.2689{]},\\
{[}-86.5283, 33.2689{]},\\
{[}-86.5283, 43.3913{]}\\
{]}\\
{]}, null, false);

Map.centerObject(geometry, 6);

function annotate(image) \{ // Annotates an image by adding outline
border and cloudiness // Cloudiness is shown as a text string rendered
at the image center. // Add an edge around the image. var edge =
ee.FeatureCollection({[}image{]})\\
.style(\{\\
color: `cccc00cc',\\
fillColor: `00000000' \}); // Draw cloudiness as text. var props = \{\\
textColor: `0000aa',\\
outlineColor: `ffffff',\\
outlineWidth: 2,\\
outlineOpacity: 0.6,\\
fontSize: 24,\\
fontType: `Consolas' \}; var center = image.geometry().centroid(1); var
str = ee.Number(image.get(`CLOUD\_COVER')).format(`\%.2f'); var scale =
Map.getScale(); var textCloudiness = text.draw(str, center, scale,
props); // Shift left 25 pixels. textCloudiness = textCloudiness\\
.translate(-scale * 25, 0, `meters', `EPSG:3857'); // Merge results.
return ee.ImageCollection({[}edge, textCloudiness{]}).mosaic();\\
\}

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Select images.  }
\KeywordTok{var}\NormalTok{ images }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_RT\_TOA\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\DecValTok{5}\OperatorTok{,} \DecValTok{4}\OperatorTok{,} \DecValTok{2}\NormalTok{])  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(geometry)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2018{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2018{-}01{-}7\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// dim background.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}black\textquotesingle{}}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}black\textquotesingle{}}\OperatorTok{,} \KeywordTok{true}\OperatorTok{,} \FloatTok{0.5}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Show images.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(images}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \FloatTok{0.05}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
   \DataTypeTok{gamma}\OperatorTok{:} \FloatTok{1.4}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}images\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Show annotations.  }
\KeywordTok{var}\NormalTok{ labels }\OperatorTok{=}\NormalTok{ images}\OperatorTok{.}\FunctionTok{map}\NormalTok{(annotate)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ labelsLayer }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Map}\OperatorTok{.}\FunctionTok{Layer}\NormalTok{(labels}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}annotations\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{add}\NormalTok{(labelsLayer)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result of defining and mapping this function over the filtered set
of images is shown in Fig. F6.0.9. Notice that by adding an outline
around the text, we can ensure the text is visible for both dark and
light images. Earth Engine requires casting properties to their
corresponding value type, which is why we've used ee.Number (as
described in Chap. F1.0) before generating a formatted string. Also, we
have shifted the resulting text image 25 pixels to the left. This was
necessary to ensure that the text is positioned properly. In more
complex text rendering applications, users may be required to compute
the text position in a different way using ee.Geometry calls from the
Earth Engine API: for example, by positioning text labels somewhere near
the corners.

\begin{figure}

{\centering \includegraphics{./F6/image3.png}

}

\caption{Fig. F6.0.9 Annotating Landsat 8 images with image boundaries,
border, and text strings indicating cloudiness}

\end{figure}

Because we render text labels using the Earth Engine raster API, they
are not automatically scaled depending on map zoom size. This may cause
unwanted artifacts; To avoid that, the text labels image needs to be
updated every time the map zoom changes. To implement this in a script,
we can make use of the Map API---in particular, the Map.onChangeZoom
event handler. The following code snippet shows how the image containing
text annotations can be re-rendered every time the map zoom changes. Add
it to the end of your script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// re{-}render (rescale) annotations when map zoom changes.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{onChangeZoom}\NormalTok{(}\KeywordTok{function}\NormalTok{(zoom) \{  }
\NormalTok{   labelsLayer}\OperatorTok{.}\FunctionTok{setEeObject}\NormalTok{(images}\OperatorTok{.}\FunctionTok{map}\NormalTok{(annotate))}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60f. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Try commenting that event handler and observe how annotation rendering
changes when you zoom in or zoom out.

\hypertarget{animations}{%
\subsection{Animations}\label{animations}}

Visualizing raster images as animations is a useful technique to explore
changes in time-dependent datasets, but also, to render short animations
to communicate how changing various parameters affects the resulting
image---for example, varying thresholds of spectral indices resulting in
different binary maps or the changing geometry of vector features.

Animations are very useful when exploring satellite imagery, as they
allow viewers to quickly comprehend dynamics of changes of earth surface
or atmospheric properties. Animations can also help to decide what steps
should be taken next to designing a robust algorithm to extract useful
information from satellite image time series. Earth Engine provides two
standard ways to generate animations: as animated GIFs, and as AVI video
clips. Animation can also be rendered from a sequence of images exported
from Earth Engine, using numerous tools such as ffmpeg or moviepy.
However, in many cases it is useful to have a way to quickly explore
image collections as animation without requiring extra steps.

In this section, we will generate animations in three different ways:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Generate animated GIF
\item
  Export video as an AVI file to Google Drive
\item
  Animate image collection interactively using UI controls and map
  layers
\end{enumerate}

We will use an image collection showing sea ice as an input dataset to
generate animations with visualization parameters from earlier. However,
instead of querying a single Sentinel-1 image, let's generate a filtered
image collection with all images intersecting with our area of interest.
After importing some packages and palettes and defining a point and
rectangle, we'll build the image collection. Here we will use point
geometry to define the location where the image date label will be
rendered and the rectangle geometry to indicate the area of interest for
the animation. To do this we will build the following logic in a new
script. Open a new script and paste the following code into it:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Include packages.  }
\KeywordTok{var}\NormalTok{ palettes }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:palettes\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ text }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:text\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ point }\OperatorTok{=} \CommentTok{/* color: \#98ff00 */}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}   \FloatTok{106.15944300895228}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{74.58262940096245}  
\NormalTok{])}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ rect }\OperatorTok{=} \CommentTok{/* color: \#d63000 */}\NormalTok{   ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{(  }
\NormalTok{       [  }
\NormalTok{           [  }
\NormalTok{               [}\OperatorTok{{-}}\FloatTok{106.19789515738981}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{74.56509549360152}\NormalTok{]}\OperatorTok{,}  
\NormalTok{               [}\OperatorTok{{-}}\FloatTok{106.19789515738981}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{74.78071448733921}\NormalTok{]}\OperatorTok{,}  
\NormalTok{               [}\OperatorTok{{-}}\FloatTok{104.98115931754606}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{74.78071448733921}\NormalTok{]}\OperatorTok{,}  
\NormalTok{               [}\OperatorTok{{-}}\FloatTok{104.98115931754606}\OperatorTok{,} \OperatorTok{{-}}\FloatTok{74.56509549360152}\NormalTok{]  }
\NormalTok{           ]  }
\NormalTok{       ]}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{false}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Lookup the ice palette.  }
\KeywordTok{var}\NormalTok{ palette }\OperatorTok{=}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{cmocean}\OperatorTok{.}\AttributeTok{Ice}\NormalTok{[}\DecValTok{7}\NormalTok{]}\OperatorTok{;}  
  
\CommentTok{// Show it in the console.  }
\NormalTok{palettes}\OperatorTok{.}\FunctionTok{showPalette}\NormalTok{(}\StringTok{\textquotesingle{}Ice\textquotesingle{}}\OperatorTok{,}\NormalTok{ palette)}\OperatorTok{;}  
  
\CommentTok{// Center map on geometry.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(point}\OperatorTok{,} \DecValTok{9}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Select S1 images for the Thwaites glacier.  }
\KeywordTok{var}\NormalTok{ images }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S1\_GRD\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(rect)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2021{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}03{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}HH\textquotesingle{}}\NormalTok{)   }\CommentTok{// Make sure we include only images which fully contain the region geometry.   .filter(ee.Filter.isContained(\{  }
\NormalTok{       leftValue}\OperatorTok{:}\NormalTok{ rect}\OperatorTok{,}  
\NormalTok{       rightField}\OperatorTok{:} \StringTok{\textquotesingle{}.geo\textquotesingle{}}\NormalTok{   \}))  }
   \OperatorTok{.}\FunctionTok{sort}\NormalTok{(}\StringTok{\textquotesingle{}system:time\_start\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Print number of images.  }
\FunctionTok{print}\NormalTok{(images}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

As you see from the last last lines of the above code, it is frequently
useful to print the number of images in an image collection: an example
of what's often known as a ``sanity check.''

Here we have used two custom geometries to configure animations: the
green pin named point, used to filter image collection and to position
text labels drawn on top of the image, and the blue rectangle rect, used
to define a bounding box for the exported animations. To make sure that
the point and rectangle geometries are shown under the Geometry Imports
in the Code Editor, you need to click on these variables in the code and
then select the Convert link.

Notice that in addition to the bounds and date filter, we have also used
a less known isContained filter to ensure that we get only images that
fully cover our region. To better understand this filter, you could try
commenting out the filter and compare the differences, observing images
with empty (masked) pixels in the resulting image collection.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60g. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Next, to simplify the animation API calls, we will generate a composite
RGB image collection out of satellite images and draw the image's
acquisition date as a label on every image, positioned within our region
geometry.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Render images.  }
\KeywordTok{var}\NormalTok{ vis }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palette}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{15}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{1}  
\NormalTok{\}}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ scale }\OperatorTok{=} \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{getScale}\NormalTok{()}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ textProperties }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{outlineColor}\OperatorTok{:} \StringTok{\textquotesingle{}000000\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{outlineWidth}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}  
   \DataTypeTok{outlineOpacity}\OperatorTok{:} \FloatTok{0.6}  
\NormalTok{\}}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ imagesRgb }\OperatorTok{=}\NormalTok{ images}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(i) \{   }\CommentTok{// Use the date as the label.   var label = i.date().format(\textquotesingle{}YYYY{-}MM{-}dd\textquotesingle{});   var labelImage = text.draw(label, point, scale,  }
\NormalTok{       textProperties)}\OperatorTok{;}   \ControlFlowTok{return}\NormalTok{ i}\OperatorTok{.}\FunctionTok{visualize}\NormalTok{(vis)  }
       \OperatorTok{.}\FunctionTok{blend}\NormalTok{(labelImage) }\CommentTok{// Blend label image on top.       .set(\{  }
           \DataTypeTok{label}\OperatorTok{:}\NormalTok{ label  }
\NormalTok{       \})}\OperatorTok{;} \CommentTok{// Keep the text property.  }
\NormalTok{\})}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(imagesRgb}\OperatorTok{.}\FunctionTok{first}\NormalTok{())}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(rect}\OperatorTok{,}\NormalTok{ \{}\DataTypeTok{color}\OperatorTok{:}\StringTok{\textquotesingle{}blue\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}rect\textquotesingle{}}\OperatorTok{,} \DecValTok{1}\OperatorTok{,} \FloatTok{0.5}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

In addition to printing the size of the ImageCollection, we also often
begin by adding a single image to the map from a mapped collection to
see that everything works as expected---another example of a sanity
check. The resulting map layer will look like F6.0.10.

\begin{figure}

{\centering \includegraphics{./F6/image6.png}

}

\caption{Fig. F6.0.10 The results of adding the first layer from the RGB
composite image collection showing Sentinel-1 images with a label
blended on top at a specified location. The blue geometry is used to
define the bounds for the animation to be exported.}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60h. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Animation 1: Animated GIF with ui.Thumbnail

The quickest way to generate an animation in Earth Engine is to use the
animated GIF API and either print it to the Console or print the URL to
download the generated GIF. The following code snippet will result in an
animated GIF as well as the URL to the animated GIF printed to Console.
This is as shown in Fig. F6.0.11:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Define GIF visualization parameters.  }
\KeywordTok{var}\NormalTok{ gifParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{region}\OperatorTok{:}\NormalTok{ rect}\OperatorTok{,}  
   \DataTypeTok{dimensions}\OperatorTok{:} \DecValTok{600}\OperatorTok{,}  
   \DataTypeTok{crs}\OperatorTok{:} \StringTok{\textquotesingle{}EPSG:3857\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{framesPerSecond}\OperatorTok{:} \DecValTok{10}  
\NormalTok{\}}\OperatorTok{;}  
  
\CommentTok{// Print the GIF URL to the console.  }
\FunctionTok{print}\NormalTok{(imagesRgb}\OperatorTok{.}\FunctionTok{getVideoThumbURL}\NormalTok{(gifParams))}\OperatorTok{;}  
  
\CommentTok{// Render the GIF animation in the console.  }
\FunctionTok{print}\NormalTok{(ui}\OperatorTok{.}\FunctionTok{Thumbnail}\NormalTok{(imagesRgb}\OperatorTok{,}\NormalTok{ gifParams))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Earth Engine provides multiple options to specify the size of the
resulting video. In this example we specify 600 as the size of the
maximum dimension. We also specify the number of frames per second for
the resulting animated GIF as well as the target projected coordinate
system to be used (EPSG:3857 here, which is the projection used in web
maps such as Google Maps and the Code Editor background).

\begin{figure}

{\centering \includegraphics{./F6/image56.png}

}

\caption{Fig. F6.0.11 Console output after running the animated GIF code
snippet, showing the GIF URL and an animation shown directly in the
Console}

\end{figure}

Animation 2: Exporting an Animation with Export.video.toDrive

Animated GIFs can be useful to generate animations quickly. However,
they have several limitations. In particular, they are limited to 256
colors, become large for larger animations, and most web players do not
provide play controls when playing animated GIFs. To overcome these
limitations, Earth Engine provides export of animations as video files
in MP4 format. Let's use the same RGB image collection we have used for
the animated GIF to generate a short video. We can ask Earth Engine to
export the video to the Google Drive using the following code snippet:

Export.video.toDrive(\{\\
collection: imagesRgb,\\
description: `ice-animation',\\
fileNamePrefix: `ice-animation',\\
framesPerSecond: 10,\\
dimensions: 600,\\
region: rect,\\
crs: `EPSG:3857'\\
\});

Here, many arguments to the Export.video.toDrive function resemble the
ones we've used in the ee.Image.getVideoThumbURL code above. Additional
arguments include description and fileNamePrefix, which are required to
configure the name of the task and the target file of the video file to
be saved to Google Drive. Running the above code will result in a new
task created under the Tasks tab in the Code Editor. Starting the export
video task (F6.0.12) will result in a video file saved in the Google
Drive once completed.

\begin{figure}

{\centering \includegraphics{./F6/image9.png}

}

\caption{Fig. F6.0.12 A new export video tasks in the Tasks panel of the
Code Editor}

\end{figure}

Animation 3: The Custom Animation Package

For the last animation example, we will use the custom package
`users/gena/packages:animation', built using the Earth Engine User
Interface API. The main difference between this package and the above
examples is that it generates an interactive animation by adding Map
layers individually to the layer set, and providing UI controls that
allow users to play animations or interactively switch between frames.
The animate function in that package generates an interactive animation
of an ImageCollection, as described below. This function has a number of
optional arguments allowing you to configure, for example, the number of
frames to be animated, the number of frames to be preloaded, or a few
others. The optional parameters to control the function are the
following:

\begin{itemize}
\tightlist
\item
  maxFrames: maximum number of frames to show (default: 30)
\item
  vis: visualization parameters for every frame (default: \{\})
\item
  Label: text property of images to show in the animation controls
  (default: undefined)
\item
  width: width of the animation panel (default: `600px')
\item
  compact: show only play control and frame slider (default: false)
\item
  position: position of the animation panel (default: `top-center')
\item
  timeStep: time step (ms) used when playing animation (default: 100)
\item
  preloadCount: number of frames (map layers) to preload (default: all)
\end{itemize}

Let's call this function to add interactive animation controls to the
current Map:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// include the animation package  }
\KeywordTok{var}\NormalTok{ animation }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{\textquotesingle{}users/gena/packages:animation\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// show animation controls  }
\NormalTok{animation}\OperatorTok{.}\FunctionTok{animate}\NormalTok{(imagesRgb}\OperatorTok{,}\NormalTok{ \{  }
 \DataTypeTok{label}\OperatorTok{:} \StringTok{\textquotesingle{}label\textquotesingle{}}\OperatorTok{,}  
 \DataTypeTok{maxFrames}\OperatorTok{:} \DecValTok{50}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Before using the interactive animation API, we need to include the
corresponding package using require. Here we provide our pre-rendered
image collection and two optional parameters (label and maxFrames). The
first optional parameter label indicates that every image in our image
collection has the `label' text property. The animate function uses this
property to name map layers as well as to visualize in the animation UI
controls when switching between frames. This can be useful when
inspecting image collections. The second optional parameter, maxFrames,
indicates that the maximum number of animation frames that we would like
to visualize is 50. To prevent the Code Editor from crashing, this
parameter should not be too large: it is best to keep it below 100. For
a much larger number of frames, it is better to use the Export video or
animated GIF API. Running this code snippet will result in the animation
control panel added to the map as shown in Fig. F6.0.13.

It is important to note that the animation API uses asynchronous UI
calls to make sure that the Code Editor does not hang when running the
script. The drawback of this is that for complex image collections, a
large amount of processing is required. Hence, it may take some time to
process all images and to visualize the interactive animation panel. The
same is true for map layer names: they are updated once the animation
panel is visualized. Also, map layers used to visualize individual
images in the provided image collection may require some time to be
rendered.

\begin{figure}

{\centering \includegraphics{./F6/image70.png}

}

\caption{Fig. F6.0.13 Interactive animation controls when using custom
animation API}

\end{figure}

The main advantage of the interactive animation API is that it provides
a way to explore image collections at frame-by-frame basis, which can
greatly improve our visual understanding of the changes captured in sets
of images.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60i. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{terrain-visualization}{%
\subsection{Terrain Visualization}\label{terrain-visualization}}

This section introduces several raster visualization techniques useful
to visualize terrain data such as:

\begin{itemize}
\tightlist
\item
  Basic hillshading and parameters (light azimuth, elevation)
\item
  Combining elevation data and colors using HSV transform (Wikipedia:
  HSL and HSV)
\item
  Adding shadows
\end{itemize}

One special type of raster data is data that represents height.
Elevation data can include topography, bathymetry, but also other forms
of height, such as sea surface height can be presented as a terrain.

Height is often visualized using the concept of directional light with a
technique called hillshading. Because height is such a common feature in
our environment, we also have an expectancy of how height is visualized.
If height is visualized using a simple grayscale colormap, it looks very
unnatural (Fig. F6.0.14, top left). By using hillshading, data
immediately looks more natural (Fig. F6.0.14, top middle).

We can further improve the visualization by including shadows (Fig.
F6.0.14, top right). A final step is to replace the simple grayscale
colormap with a perceptual uniform topographic colormap and mix this
with the hillshading and shadows (Fig. F6.0.14, bottom). This section
explains how to apply these techniques.

We'll focus on elevation data stored in raster form. Elevation data is
not always stored in raster formats. Other data formats include
Triangulated Irregular Network (TIN), which allows storing information
at varying resolutions and as 3D objects. This format allows one to have
overlapping geometries, such as bridges with a road below it. In
raster-based digital elevation models, in contrast, there can only be
one height recorded for each pixel.

Let's start by loading data from a digital elevation model. This loads a
topographic dataset from the Netherlands (Algemeen Hoogtebestand
Nederland). It is a Digital Surface Model, based on airborne LIDAR
measurements regridded to 0.5 m resolution. Enter the following code in
a new script.

var dem = ee.Image(`AHN/AHN2\_05M\_RUW');

We can visualize this dataset using a sequential gradient colormap from
black to white. This results in Fig. F6.0.14. One can infer which areas
are lower and which are higher, but the visualization does not quite
``feel'' like a terrain.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Change map style to HYBRID and center map on the Netherlands  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{\textquotesingle{}HYBRID\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\FloatTok{4.4082}\OperatorTok{,} \FloatTok{52.1775}\OperatorTok{,} \DecValTok{18}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Visualize DEM using black{-}white color palette  }
\KeywordTok{var}\NormalTok{ palette }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}black\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}white\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ demRGB }\OperatorTok{=}\NormalTok{ dem}\OperatorTok{.}\FunctionTok{visualize}\NormalTok{(\{  }
   \DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\DecValTok{5}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{5}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palette  }
\NormalTok{\})}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(demRGB}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}\StringTok{\textquotesingle{}DEM\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

An important step to visualize terrain is to add shadows created by a
distant point source of light. This is referred to as hillshading or a
shaded relief map. This type of map became popular in the 1940s through
the work of Edward Imhoff, who also used grayscale colormaps (Imhoff,
2015). Here we'll use the `gena/packages:utils' library to combine the
colormap image with the shadows. That Earth Engine package implements a
hillshadeRGB function to simplify rendering of images enhanced with
hillshading and shadow effects. One important argument this function
takes is the light azimuth---an angle from the image plane upward to the
light source (the Sun). This should always be set to the top left to
avoid bistable perception artifacts, in which the DEM can be
misperceived as inverted.

var utils = require(`users/gena/packages:utils');

var weight = 0.4; // Weight of Hillshade vs RGB (0 - flat, 1 -
hillshaded).\\
var exaggeration = 5; // Vertical exaggeration.\\
var azimuth = 315; // Sun azimuth.\\
var zenith = 20; // Sun elevation.\\
var brightness = -0.05; // 0 - default.\\
var contrast = 0.05; // 0 - default.\\
var saturation = 0.8; // 1 - default.\\
var castShadows = false;

var rgb = utils.hillshadeRGB(\\
demRGB, dem, weight, exaggeration, azimuth, zenith,\\
contrast, brightness, saturation, castShadows);

Map.addLayer(rgb, \{\}, `DEM (no shadows)');

Standard hillshading only determines per pixel if it will be directed to
the light or not. One can also project shadows on the map. That is done
using the ee.Algorithms.HillShadow algorithm. Here we'll turn on
castShadows in the hillshadeRGB function. This results in a more
realistic map, as can be seen in Figure F6.0.14.

var castShadows = true;

var rgb = utils.hillshadeRGB(\\
demRGB, dem, weight, exaggeration, azimuth, zenith,\\
contrast, brightness, saturation, castShadows);

Map.addLayer(rgb, \{\}, `DEM (with shadows)');

The final step is to add a topographic colormap. To visualize
topographic information, one often uses special topographic colormaps.
Here we'll use the oleron colormap from crameri. The colors get mixed
with the shadows using the hillshadeRGB function. As you can see in Fig.
F6.0.14, this gives a nice overview of the terrain. The area colored in
blue is located below sea level.

var palettes = require(`users/gena/packages:palettes');\\
var palette = palettes.crameri.oleron{[}50{]};

var demRGB = dem.visualize(\{\\
min: -5,\\
max: 5,\\
palette: palette\\
\});

var castShadows = true;

var rgb = utils.hillshadeRGB(\\
demRGB, dem, weight, exaggeration, azimuth, zenith,\\
contrast, brightness, saturation, castShadows);

Map.addLayer(rgb, \{\}, `DEM colormap');

Steps to further improve a terrain visualization include using light
sources from multiple directions. This allows the user to render terrain
to appear more natural. In the real world light is often scattered by
clouds and other reflections.

One can also use lights to emphasize certain regions. To use even more
advanced lighting techniques one can use a raytracing engine, such as
the R rayshader library, as discussed earlier in this chapter. The
raytracing engine in the Blender 3D program is also capable of producing
stunning terrain visualizations using physical-based rendering, mist,
environment lights, and camera effects such as depth of field.

\includegraphics{./F6/image36.png}\includegraphics{./F6/image58.png}\includegraphics{./F6/image47.png}

\includegraphics{./F6/image55.png}

Figure F6.0.14 Hillshading with shadows

Steps in visualizing a topographic dataset:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Top left, topography with grayscale colormap
\item
  Top middle, topography with grayscale colormap and hillshading
\item
  Top right, topography with grayscale colormap, hillshading, and
  shadows
\item
  Bottom, topography with topographic colormap, hillshading, and shadows
\end{enumerate}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F60j. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{conclusion-21}{%
\subsection*{Conclusion}\label{conclusion-21}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter we have learned about several techniques that can
greatly improve visualization and analysis of images and image
collections. Using predefined palettes can help to better comprehend and
communicate Earth observation data, and combining with other
visualization techniques such as hillshading and annotations can help to
better understand processes studied with Earth Engine. When working with
image collections, it is often very helpful to analyze their properties
through time by visualizing them as animations. Usually, this step helps
to better understand dynamics of the changes that are stored in image
collections and to develop a proper algorithm to study these changes.

\hypertarget{references-14}{%
\subsection*{References}\label{references-14}}
\addcontentsline{toc}{subsection}{References}

Burrough PA, McDonnell RA, Lloyd CD (2015) Principles of Geographical
Information Systems. Oxford University Press

Crameri F, Shephard GE, Heron PJ (2020) The misuse of colour in science
communication. Nat Commun 11:1--10.
https://doi.org/10.1038/s41467-020-19160-7

Imhof E (2015) Cartographic Relief Presentation. Walter de Gruyter GmbH
\& Co KG

Lhermitte S, Sun S, Shuman C, et al (2020) Damage accelerates ice shelf
instability and mass loss in Amundsen Sea Embayment. Proc Natl Acad Sci
USA 117:24735--24741. https://doi.org/10.1073/pnas.1912890117

Thyng KM, Greene CA, Hetland RD, et al (2016) True colors of
oceanography. Oceanography 29:9--13

Wikipedia (2022) Terrain cartography.
https://en.wikipedia.org/wiki/Terrain\_cartography\#Shaded\_relief.
Accessed 1 Apr 2022

Wikipedia (2022) HSL and HSV.
https://en.wikipedia.org/wiki/HSL\_and\_HSV. Accessed 1 Apr 2022

Wild CT, Alley KE, Muto A, et al (2022) Weakening of the pinning point
buttressing Thwaites Glacier, West Antarctica. Cryosphere 16:397--417.
https://doi.org/10.5194/tc-16-397-2022

Wilkinson L (2005) The Grammar of Graphics. Springer Verlag

\hypertarget{collaborating-in-earth-engine-with-scripts-and-assets}{%
\section{Collaborating in Earth Engine with Scripts and
Assets}\label{collaborating-in-earth-engine-with-scripts-and-assets}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-20}{%
\subsubsection*{Author}\label{author-20}}

Sabrina H. Szeto

\hypertarget{overview-22}{%
\subsubsection*{Overview}\label{overview-22}}

Many users find themselves needing to collaborate with others in Earth
Engine at some point. Students may need to work on a group project,
people from different organizations might want to collaborate on
research together, or people may want to share a script or an asset they
created with others. This chapter will show you how to collaborate with
others and share your work.

\hypertarget{learning-outcomes-22}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-22}}

\begin{itemize}
\tightlist
\item
  Understanding when it is important to share a script or asset.
\item
  Understanding what roles and permission options are available.
\item
  Sharing a script with others.
\item
  Sharing an asset with others.
\item
  Sharing an asset so it can be displayed in an app.
\item
  Sharing a repository with others.
\item
  Seeing who made changes to a script and what changes were made.
\item
  Reverting to a previous version of a script.
\item
  Using the require function to load modules.
\item
  Creating a script to share as a module.
\end{itemize}

\hypertarget{assumes-you-know-how-to-22}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-22}}

\begin{itemize}
\tightlist
\item
  Sign up for an Earth Engine account, open the Code Editor, and save
  your script (Chap. F1.0).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-18}{%
\subsection*{Introduction}\label{introduction-18}}

Many people find themselves needing to share a script when they
encounter a problem; they wish to share the script with someone else so
they can ask a question. When this occurs, sharing a link to the script
often suffices. The other person can then make comments or changes
before sending a new link to the modified script.

If you have included any assets from the Asset Manager in your script,
you will also need to share these assets in order for your script to
work for your colleague. The same goes for sharing assets to be
displayed in an app.

Another common situation involves collaborating with others on a
project. They may have some scripts they have written that they want to
reuse or modify for the new project. Alternatively, several people might
want to work on the same script together. For this situation, sharing a
repository would be the best way forward; team members will be able to
see who made what changes to a script and even revert to a previous
version.

If you or your group members find yourselves repeatedly reusing certain
functions for visualization or for part of your analysis, you could use
the require module to call that function instead of having to copy and
paste it into a new script each time. You could even make this function
or module available to others to use via require.

Let's get started. For this lab, you will need to work in small groups
or pairs.

\hypertarget{using-get-link-to-share-a-script}{%
\subsection{Using Get Link to Share a
Script}\label{using-get-link-to-share-a-script}}

Copy and paste the following code into the Code Editor.

print(`The author of this script is MyName.');

Replace MyName with your name, then click on Save to save the script in
your home repository. Next, click on the Get Link button and copy the
link to this script onto your clipboard. Using your email program of
choice, send this script to one of your group members.

Now add the following code below the line of code that you pasted
earlier.

print(`I just sent this script to GroupMemberName.');

Replace GroupMemberName with the name of the person you sent this script
to, then save the script again. Next, click on the Get Link button and
copy the link to this script onto your clipboard. Using your email
program of choice, send this script to the same person.

Question 1. You should also have received two emails from someone in
your group who is also doing this exercise. Open the first and second
links in your Code Editor by clicking on them. Is the content of both
scripts the same?

Answer: No, the scripts will be different, because Get Link sends a
snapshot of the script at a particular point in time. Thus, even though
the script was updated, the first link does not reflect that change.

Question 2. What happens when you check the box for Hide code panel or
Disable auto-run before sharing the script?

Answer. Hide code panel will minimize the code panel so the person you
send the script to will see the Map maximized. This is useful when you
want to draw the person's attention to the results rather than to the
code. To expand the code panel, they have to click on the Show code
button. Disable auto-run is helpful when you do not want the script to
start running when the person you sent it to opens it. Perhaps your
script takes very long to run or requires particular user inputs and you
just want to share the code with the person.

\hypertarget{sharing-assets-from-your-asset-manager}{%
\subsection{Sharing Assets from Your Asset
Manager}\label{sharing-assets-from-your-asset-manager}}

When you clicked the Get Link button earlier, you may have noticed a
note in the popup reading: ``To give others access to assets in the code
snapshot, you may need to share them.'' If your script uses an asset
that you have uploaded into your Asset Manager, you will need to share
that asset as well. If not, an error message will appear when the person
you shared the script with tries to run it.

Before sharing an asset, think about whether you have permission to
share it. Is this some data that is owned by you, or did you get it from
somewhere else? Do you need permission to share this asset? Make sure
you have the permission to share an asset before doing so.

Now, let's practice sharing assets. First, navigate to your Asset
Manager by clicking on the Assets tab in the left panel. If you already
have some assets uploaded, pick one that you have permission to share.
If not, upload one to your Asset Manager. If you don't have a shapefile
or raster to upload, you can upload a small text file. Consult the Earth
Engine documentation for how to do this; it will take only a few steps.

Hover your cursor over that asset in your Asset Manager. The asset gets
highlighted in gray and three buttons appear to the right of the asset.
Click on the first button from the left (outlined in red in Fig.
F6.1.1). This icon means ``share.''

\begin{figure}

{\centering \includegraphics{./F6/image26.png}

}

\caption{Fig. F6.1.1 Three assets in the Asset Manager}

\end{figure}

After you click the share button, a Share Image popup will appear (Fig.
F6.1.2). This popup contains information about the path of the asset and
the email address of the owner. The owner of the asset can decide who
can view and edit the asset.

Click on the dropdown menu outlined in red in Fig. F6.1.2. You will see
two options for permissions: Reader and Writer. A Reader can view the
asset, while a Writer can both view and make changes to it. For example,
a Writer could add a new image to an ImageCollection. A Writer can also
add other people to view or edit the asset, and a Writer can delete the
asset. When in doubt, give someone the Reader role rather than the
Writer role.

\begin{figure}

{\centering \includegraphics{./F6/image10.png}

}

\caption{Fig. F6.1.2 The Share Image popup window}

\end{figure}

To share an asset with someone, you can type their email address into
the Email or domain text field, choose Reader or Writer in the dropdown
menu, and then click on Add Access. You can also share an asset with
everyone with a certain email domain, which is useful if you want to
share an asset with everyone in your organization, for instance.

If you want to share reading access publicly, then check the box that
says Anyone can read. Note that you still need to share the link to the
asset in order for others to access it. The only exceptions to this are
when you are using the asset in a script and sharing that script using
the Get Link button or when you share the asset with an Earth Engine
app. To do the latter, use the Select an app dropdown menu (outlined in
orange in Fig. F6.1.2) and click Add App Access.

Question 3. Share an asset with a group member and give them reader
access. Send them the link to that asset. You will also receive a link
from someone else in your group. Open that link. What can you do with
that asset? What do you need to do to import it into a script?

Answer: You can view details about the asset and import it for use in a
script in the Code Editor. To import the asset, click on the blue Import
button.

Question 4. Share an asset with a group member and give them writer
access. Send them the link to that asset. You will also receive a link
from someone else in your group. Open that link. What can you do with
that asset? Try sharing the asset with a different group member.

Answer: You can view details about the asset and import it for use in a
script in the Code Editor. You can also share the asset with others and
delete the asset.

\hypertarget{working-with-shared-repositories}{%
\subsection{Working with Shared
Repositories}\label{working-with-shared-repositories}}

Now that you know how to share assets and scripts, let's move on to
sharing repositories. In this section, you will learn about different
types of repositories and how to add a repository that someone else
shared with you. You will also learn how to view previous versions of a
script and how to revert back to an earlier version.

Earlier, we learned how to share a script using the Get Link button.
This link shares a code snapshot from a script. This snapshot does not
reflect any changes made to the script after the time the link was
shared. If you want to share a script that updates to reflect the most
current version when it is opened, you need to share a repository with
that script instead.

If you look under the Scripts tab of the leftmost panel in the Code
Editor, you will see that the first three categories are labeled Owner,
Reader, and Writer.

\begin{itemize}
\tightlist
\item
  Repositories categorized under Owner are created and owned by you. No
  one else has access to view or make changes to them until you share
  these repositories.
\item
  Repositories categorized under Reader are repositories to which you
  have reader access. You can view the scripts but not make any changes
  to them. If you want to make any changes, you will need to save the
  script as a new file in a repository that you own.
\item
  Repositories categorized under Writer are repositories to which you
  have writer access. This means you can view and make changes to the
  scripts.
\end{itemize}

Let's practice creating and sharing repositories. We will start by
making a new repository. Click on the red New button located in the left
panel. Select Repository from the dropdown menu. A New repository popup
window will open (Fig. F6.1.3).

\begin{figure}

{\centering \includegraphics{./F6/image25.png}

}

\caption{Fig. F6.1.3 The New repository popup window}

\end{figure}

In the popup window's text field, type a name for your new repository,
such as ``ForSharing1,'' then click on the blue Create button. You will
see the new repository appear under the Owner category in the Scripts
tab (Fig. F6.1.4).

Now, share this new repository with your group members: Hover your
cursor over the repository you want to share. The repository gets
highlighted in gray, and three buttons appear. Click on the Gear icon
(outlined in red in Fig. F6.1.4).

\begin{figure}

{\centering \includegraphics{./F6/image71.png}

}

\caption{Fig. F6.1.4 Three repositories under the Owner category}

\end{figure}

A Share Repo popup window appears (Fig. F6.1.5) which is very similar to
the Share Image popup window we saw in Fig. F6.1.2. The method for
sharing a repository with a specific user or the general public is the
same as for sharing assets.

Type the email address of a group member in the Email or domain text
field and give this person a writer role by selecting Writer in the
dropdown menu, then click on Add Access.

\begin{figure}

{\centering \includegraphics{./F6/image29.png}

}

\caption{Fig. F6.1.5. The Share Repo popup window}

\end{figure}

Your group member should receive an email inviting them to edit the
repository. Check your email inbox for the repository that your group
member has shared with you. When you open that email, you will see
content similar to what is shown in Fig. F6.1.6.

\begin{figure}

{\centering \includegraphics{./F6/image50.png}

}

\caption{Fig. F6.1.6 The ``Invitation to edit'' email}

\end{figure}

Now, click on the blue button that says Add {[}repository path{]} to
your Earth Engine Code Editor. You will find the new repository added to
the Writer category in your Scripts tab. The repository path will
contain the username of your group member, such as
users/username/sharing.

Now, let's add a script to the empty repository. Click on the red New
button in the Scripts tab and select File from the dropdown menu. A
Create file popup will appear, as shown in Fig. F6.1.7. Click on the
gray arrow beside the default path to open a dropdown menu that will
allow you to choose the path of the repository that your group member
shared with you. Type a new File Name in the text field, such as
``exercise,'' then click on the blue OK button to create the file.

\begin{figure}

{\centering \includegraphics{./F6/image69.png}

}

\caption{Fig. F6.1.7 The Create file popup window}

\end{figure}

A new file should now appear in the shared repository in the Writer
category. If you don't see it, click on the Refresh icon, which is to
the right of the red New button in the Scripts tab.

Double-click on the new script in the shared repository to open it.
Then, copy and paste the following code to your Code Editor.

print(`The owner of this repository is GroupMemberName.');

Replace GroupMemberName with the name of your group member, then click
Save to save the script in the shared repository, which is under the
Writer category.

Now, navigate to the repository under Owner which you shared with your
group member. Open the new script which they just created by
double-clicking it.

Add the following code below the line of code that you pasted earlier.

print(`This script is shared with MyName.');

Replace MyName with your name, then save the script.

Next, we will compare changes made to the script. Click on the Versions
icon (outlined in red in Fig. F6.1.8).

\begin{figure}

{\centering \includegraphics{./F6/image17.png}

}

\caption{Fig. F6.1.8 Changes made and previous versions of the script}

\end{figure}

A popup window will appear, titled Revision history, followed by the
path of the script (Fig. F6.1.9). There are three columns of information
below the title.

\begin{itemize}
\tightlist
\item
  The left column contains the dates on which changes have been made.
\item
  The middle column contains the usernames of the people who made
  changes.
\item
  The right column contains information about what changes were made.
\end{itemize}

The most recent version of the script is shown in the first row, while
previous versions are listed in subsequent rows. (More advanced users
may notice that this is actually a Git repository.)

\begin{figure}

{\centering \includegraphics{./F6/image19.png}

}

\caption{Fig. F6.1.9 The Revision history popup window}

\end{figure}

If you hover your cursor over a row, the row will be highlighted in gray
and a button labeled Compare will appear. Clicking on this button allows
you to compare differences between the current version of the script and
a previous version in a Version comparison popup window (Fig. F6.1.10).

\begin{figure}

{\centering \includegraphics{./F6/image30.png}

}

\caption{Fig. F6.1.10 The Version comparison popup window}

\end{figure}

In the Version comparison popup, you will see text highlighted in two
different colors. Text highlighted in red shows code that was present in
the older version but is absent in the current version (the ``latest
commit''). Text highlighted in green shows code that is present in the
current version but that was absent in the older version. Generally
speaking, text highlighted in red has been removed in the current
version and text highlighted in green has been added to the current
version. Text that is not highlighted shows code that is present in both
versions.

Question 5. What text, if any, is highlighted in red when you click on
Compare in your ``exercise'' script?

Answer: No text is highlighted in red, because none was removed between
the previous and current versions of the script.

Question 6. What text, if any, is highlighted in green when you click on
Compare in your ``exercise'' script?

Answer: print(`This script is shared with MyName.');

Question 7. What happens when you click on the blue Revert button?

Answer: The script reverts to the previous version, in which the only
line of code is

print(`The owner of this repository is GroupMemberName.');

\hypertarget{using-the-require-function-to-load-a-module}{%
\subsection{Using the Require Function to Load a
Module}\label{using-the-require-function-to-load-a-module}}

In earlier chapters, you may have noticed that the require function
allows you to reuse code that has already been written without having to
copy and paste it into your current script. For example, you might have
written a function for cloud masking that you would like to use in
multiple scripts. Saving this function as a module enables you to share
the code across your own scripts and with other people. Or you might
discover a new module with capabilities you need written by other
authors. This section will show you how to use the require function to
create and share your own module or to load a module that someone else
has shared.

The module we will use is ee-palettes, which enables users to visualize
raster data using common specialized color palettes (Donchyts et
al.~2019). (If you would like to learn more about using these color
palettes, the ee-palettes module is described and illustrated in detail
in Chap. F6.0.) The first step is to go to this link to accept access to
the repository as a reader:
\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dusers/gena/packages\&sa=D\&source=editors\&ust=1671458841147867\&usg=AOvVaw2lfbVvfKSe6Nym_B5h25Z7}{https://code.earthengine.google.com/?accept\_repo=users/gena/}\href{https://www.google.com/url?q=https://code.earthengine.google.com/?accept_repo\%3Dusers/gena/packages\&sa=D\&source=editors\&ust=1671458841148247\&usg=AOvVaw396ktWqHZ6LRi90IQjOwxZ}{packages}

Now, if you navigate to your Reader directory in the Code Editor, you
should see a new repository called `users/gena/packages' listed. Look
for a script called `palettes' and click on it to load it in your Code
Editor.

If you scroll down, you will see that the script contains a nested
series of dictionaries with lists of hexadecimal color specifications
(as described in Chap. F2.1) that describe a color palette, as shown in
the code block below. For example, the color palette named ``Algae''
stored in the cmocean variable consists of seven colors, ranging from
dark green to light green (Fig. F6.1.11).

exports.cmocean = \{\\
Thermal: \{ 7: {[}`042333', `2c3395', `744992', `b15f82', `eb7958',
`fbb43d', `e8fa5b' {]}\\
\},\\
Haline: \{ 7: {[}`2a186c', `14439c', `206e8b', `3c9387', `5ab978',
`aad85c', `fdef9a' {]}\\
\},\\
Solar: \{ 7: {[}`331418', `682325', `973b1c', `b66413', `cb921a',
`dac62f', `e1fd4b' {]}\\
\},\\
Ice: \{ 7: {[}`040613', `292851', `3f4b96', `427bb7', `61a8c7',
`9cd4da', `eafdfd' {]}\\
\},\\
Gray: \{ 7: {[}`000000', `232323', `4a4a49', `727171', `9b9a9a',
`cacac9', `fffffd' {]}\\
\},\\
Oxy: \{ 7: {[}`400505', `850a0b', `6f6f6e', `9b9a9a', `cbcac9',
`ebf34b', `ddaf19' {]}\\
\},\\
Deep: \{ 7: {[}`fdfecc', `a5dfa7', `5dbaa4', `488e9e', `3e6495',
`3f396c', `281a2c' {]}\\
\},\\
Dense: \{ 7: {[}`e6f1f1', `a2cee2', `76a4e5', `7871d5', `7642a5',
`621d62', `360e24' {]}\\
\},\\
Algae: \{ 7: {[}`d7f9d0', `a2d595', `64b463', `129450', `126e45',
`1a482f', `122414' {]}\\
\},\\
\ldots{}\\
\}

Notice that the variable is named exports.cmocean. Adding exports to the
name of a function or variable makes it available to other scripts to
use, as it gets added to a special global variable (Chang 2017).

\begin{figure}

{\centering \includegraphics{./F6/image52.png}

}

\caption{Fig. F6.1.11 Some of the color palettes from the ee-palettes
GitHub repository}

\end{figure}

To see all the color palettes available in this module, go to
\href{https://www.google.com/url?q=https://github.com/gee-community/ee-palettes\&sa=D\&source=editors\&ust=1671458841155957\&usg=AOvVaw30bltn2S4_BDlhyuKIvkZH}{https://github.com/gee-community/ee-palettes}.

Now let's try using the ee-palettes module. Look for a script in the
same repository called `palettes-test' and click on it to load it in
your Code Editor. When you run the script, you will see digital
elevation data from the National Aeronautics and Space Administration
Shuttle Radar Topography Mission satellite visualized using two
palettes, colorbrewer.Blues and cmocean.Algae. The map will have two
layers that show the same data with different palettes.

The script first imports the digital elevation model data in the Imports
section of the Code Editor.

var dem = ee.Image(`USGS/SRTMGL1\_003');

The script then loads the ee-palettes module by using the require
function. The path to the module, `users/gena/packages:palettes', is
passed to the function. The require function is then stored in a
variable named `palettes', which will be used later to obtain the
palettes for data visualization.

var palettes = require(`users/gena/packages:palettes');

As described by Donchyts et al.~(2019), ``Each palette is defined by a
group and a name, which are separated by a period (JS object dot
notation), and a color level. To retrieve a desired palette, use JS
object notation to specify the group, name, and number of color
levels.'' We define the color palette Algae as
palettes.cmocean.Algae{[}7{]} because it is part of the group cmocean
and has 7 color levels. In the next code block, you can see that the
palettes (i.e., lists of hex colors) have been defined for use by
setting them as the value for the palette key in the visParams object
supplied to the Map.addLayer function.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// colorbrewer  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(dem}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{3000}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{colorbrewer}\OperatorTok{.}\AttributeTok{Blues}\NormalTok{[}\DecValTok{9}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}colorbrewer Blues[9]\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// cmocean  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(dem}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{3000}\OperatorTok{,}  
   \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palettes}\OperatorTok{.}\AttributeTok{cmocean}\OperatorTok{.}\AttributeTok{Algae}\NormalTok{[}\DecValTok{7}\NormalTok{]  }
\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}cmocean Algae[7]\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Question 8. Try adding a third layer to the Map with a different color
palette from ee-palettes. How easy was it to do?

Now that you have loaded and used a module shared by someone else, you
can try your hand at creating your own module and sharing it with
someone else in your group. First, go to the shared repository that you
created in Sect. 3, create a new script in that repository, and name it
``cloudmasking.''

Then, go to the Examples repository at the bottom of the Scripts tab and
select a function from the Cloud Masking repository. Let's use the
Landsat8 Surface Reflectance cloud masking script as an example. In that
script, you will see the code shown in the block below. Copy all of it
into your empty script.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// This example demonstrates the use of the Landsat 8 Collection 2, Level 2  }
\CommentTok{// QA\_PIXEL band (CFMask) to mask unwanted pixels.  }
\KeywordTok{function} \FunctionTok{maskL8sr}\NormalTok{(image) \{   }\CommentTok{// Bit 0 {-} Fill   // Bit 1 {-} Dilated Cloud   // Bit 2 {-} Cirrus   // Bit 3 {-} Cloud   // Bit 4 {-} Cloud Shadow   var qaMask = image.select(\textquotesingle{}QA\_PIXEL\textquotesingle{}).bitwiseAnd(parseInt(\textquotesingle{}11111\textquotesingle{},       2)).eq(0);   var saturationMask = image.select(\textquotesingle{}QA\_RADSAT\textquotesingle{}).eq(0);   // Apply the scaling factors to the appropriate bands.   var opticalBands = image.select(\textquotesingle{}SR\_B.\textquotesingle{}).multiply(0.0000275).add({-}       0.2);   var thermalBands = image.select(\textquotesingle{}ST\_B.*\textquotesingle{}).multiply(0.00341802)  }
       \OperatorTok{.}\FunctionTok{add}\NormalTok{(}\FloatTok{149.0}\NormalTok{)}\OperatorTok{;}   \CommentTok{// Replace the original bands with the scaled ones and apply the masks.   return image.addBands(opticalBands, null, true)  }
       \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(thermalBands}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{true}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(qaMask)  }
       \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(saturationMask)}\OperatorTok{;}  
\NormalTok{\}  }
  
\CommentTok{// Map the function over one year of data.  }
\KeywordTok{var}\NormalTok{ collection }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(maskL8sr)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ composite }\OperatorTok{=}\NormalTok{ collection}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the results.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\OperatorTok{{-}}\FloatTok{4.52}\OperatorTok{,} \FloatTok{40.29}\OperatorTok{,} \DecValTok{7}\NormalTok{)}\OperatorTok{;} \CommentTok{// Iberian Peninsula  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(composite}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.3}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that this code is well commented and has a header that describes
what the script does. Don't forget to comment your code and describe
what you are doing each step of the way. This is a good practice for
collaborative coding and for your own future reference.

Imagine that you changed this maskL8sr function slightly for some reason
and want to make it available to other users and scripts. To do that,
you can turn the function into a module. Copy and modify the code from
the example code into the new script you created called
``cloudmasking.'' (Hint: Store the function in a variable starting with
exports. Be careful that you don't accidentally use Export, which is
used to export datasets.)

Your script should be similar to the following code.

exports.maskL8sr = function(image) \{ // Bit 0 - Fill // Bit 1 - Dilated
Cloud // Bit 2 - Cirrus // Bit 3 - Cloud // Bit 4 - Cloud Shadow var
qaMask = image.select(`QA\_PIXEL').bitwiseAnd(parseInt( `11111',
2)).eq(0); var saturationMask = image.select(`QA\_RADSAT').eq(0); //
Apply the scaling factors to the appropriate bands. var opticalBands =
image.select(`SR\_B.').multiply(0.0000275)\\
.add(-0.2); var thermalBands =
image.select('ST\_B.*').multiply(0.00341802)\\
.add(149.0); // Replace the original bands with the scaled ones and
apply the masks. return image.addBands(opticalBands, null, true)\\
.addBands(thermalBands, null, true)\\
.updateMask(qaMask)\\
.updateMask(saturationMask);\\
\}

Next, you will create a test script that makes use of the cloud masking
module you just made. Begin by creating a new script in your shared
repository called ``cloudmasking-test.'' You can modify the last part of
the example cloud masking script to use your module.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Map the function over one year of data.  }
\KeywordTok{var}\NormalTok{ collection }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(maskL8sr)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ composite }\OperatorTok{=}\NormalTok{ collection}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the results.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\OperatorTok{{-}}\FloatTok{4.52}\OperatorTok{,} \FloatTok{40.29}\OperatorTok{,} \DecValTok{7}\NormalTok{)}\OperatorTok{;} \CommentTok{// Iberian Peninsula  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(composite}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.3}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Question 9. How will you modify the cloud masking script to use your
module? What does the script look like?

Answer: Your code might look something like the code block below.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load the module  }
\KeywordTok{var}\NormalTok{ myCloudFunctions }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(   }\StringTok{\textquotesingle{}users/myusername/my{-}shared{-}repo:cloudmasking\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Map the function over one year of data.  }
\KeywordTok{var}\NormalTok{ collection }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}LANDSAT/LC08/C02/T1\_L2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}\StringTok{\textquotesingle{}2020{-}01{-}01\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}2021{-}01{-}01\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{map}\NormalTok{(myCloudFunctions}\OperatorTok{.}\AttributeTok{maskL8sr}\NormalTok{)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ composite }\OperatorTok{=}\NormalTok{ collection}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the results.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\OperatorTok{{-}}\FloatTok{4.52}\OperatorTok{,} \FloatTok{40.29}\OperatorTok{,} \DecValTok{7}\NormalTok{)}\OperatorTok{;} \CommentTok{// Iberian Peninsula  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(composite}\OperatorTok{,}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}SR\_B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}SR\_B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \FloatTok{0.3}  
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\hypertarget{conclusion-22}{%
\subsection*{Conclusion}\label{conclusion-22}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to collaborate with others in the Earth
Engine Code Editor through sharing scripts, assets, and repositories.
You learned about different roles and permissions available for sharing
and when it is appropriate to use each. In addition, you are now able to
see what changes have been made to a script and revert to a previous
version. Lastly, you loaded and used a module that was shared with you
and created your own module for sharing. You are now ready to start
collaborating and developing scripts with others.

\hypertarget{references-15}{%
\subsection*{References}\label{references-15}}
\addcontentsline{toc}{subsection}{References}

Chang A (2017) Making it easier to reuse code with Earth Engine script
modules. In: Google Earth and Earth Engine.
https://medium.com/google-earth/making-it-easier-to-reuse-code-with-earth-engine-script-modules-2e93f49abb13.
Accessed 24 Feb 2022

Donchyts G, Baart F, Braaten J (2019) ee-palettes.
https://github.com/gee-community/ee-palettes. Accessed 24 Feb 2022

\hypertarget{scaling-up-in-earth-engine}{%
\section{Scaling Up in Earth Engine}\label{scaling-up-in-earth-engine}}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-21}{%
\subsubsection*{Author}\label{author-21}}

Jillian M. Deines, Stefania Di Tommaso, Nicholas Clinton, Noel Gorelick

\hypertarget{overview-23}{%
\subsubsection*{Overview}\label{overview-23}}

Commonly, when Earth Engine users move from tutorials to developing
their own processing scripts, they encounter the dreaded error messages,
``computation timed out'' or ``user memory limit exceeded.''
Computational resources are never unlimited, and the team at Earth
Engine has designed a robust system with built-in checks to ensure
server capacity is available to everyone. This chapter will introduce
general tips for creating efficient Earth Engine workflows that
accomplish users' ambitious research objectives within the constraints
of the Earth Engine ecosystem. We use two example case studies: 1)
extracting a daily climate time series for many locations across two
decades, and 2) generating a regional, cloud-free median composite from
Sentinel-2 imagery.

\hypertarget{learning-outcomes-23}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-23}}

\begin{itemize}
\tightlist
\item
  Understanding constraints on Earth Engine resource use.
\item
  Becoming familiar with multiple strategies to scale Earth Engine
  operations.
\item
  Managing large projects and multistage workflows.
\item
  Recognizing when using the Python API may be advantageous to execute
  large batches of tasks.
\end{itemize}

\hypertarget{assumes-you-know-how-to-23}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-23}}

\begin{itemize}
\tightlist
\item
  Import images and image collections, filter, and visualize (Part F1).
\item
  Write a function and map it over an ImageCollection (Chap. F4.0).
\item
  Export and import results as Earth Engine assets (Chap. F5.0).
\item
  Understand distinctions among Image, ImageCollection, Feature and
  FeatureCollection Earth Engine objects (Part F1, Part F2, Part F5).
\item
  Use the require function to load code from existing modules (Chap.
  F6.1).
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-19}{%
\subsection*{Introduction}\label{introduction-19}}

Parts F1--F5 of this book have covered key remote sensing concepts and
demonstrated how to implement them in Earth Engine. Most exercises have
used local-scale examples to enhance understanding and complete tasks
within a class-length time period. But Earth Engine's power comes from
its scalability---the ability to apply geospatial processing across
large areas and many years.

How we go from small to large scales is influenced by Earth Engine's
design. Earth Engine runs on many individual computer servers, and its
functions are designed to split up processing onto these servers. This
chapter focuses on common approaches to implement large jobs within
Earth Engine's constraints. To do so, we first discuss Earth Engine's
underlying infrastructure to provide context for existing limits. We
then cover four core concepts for scaling:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Using best coding practices.
\item
  Breaking up jobs across time.
\item
  Breaking up jobs across space.
\item
  Building a multipart workflow and exporting intermediate assets.
\end{enumerate}

\hypertarget{earth-engine-under-the-hood}{%
\subsubsection{Earth Engine: Under the
Hood}\label{earth-engine-under-the-hood}}

As you use Earth Engine, you may begin to have questions about how it
works and how you can use that knowledge to optimize your workflow. In
general, the inner workings are opaque to users. Typical fixes and
approaches that data scientists use to manage memory constraints often
don't apply. It's helpful to know what users can and cannot control, and
how your scripts translate to Earth Engine's server operations.

Earth Engine is a parallel, distributed system (see Gorelick et
al.~2017), which means that when you submit tasks, it breaks up pieces
of your query onto different processors to complete them more
efficiently. It then collects the results and returns them to you. For
many users, not having to manually design this parallel, distributed
processing is a huge benefit. For some advanced users, it can be
frustrating to not have better control. We'd argue that leaving the
details up to Earth Engine is a huge time-saver for most cases, and
learning to work within a few constraints is a good time investment.

One core concept useful to master is the relationship between
client-side and server-side operations. Client-side operations are
performed within your browser (for the JavaScript API Code Editor) or
local system (for the Python API). These include things such as
manipulating strings or numbers in JavaScript. Server-side operations
are executed on Google's servers and include all of the ee.* functions.
By using the Earth Engine APIs---JavaScript or Python---you are building
a chain of commands to send to the servers and later receive the result
back. As much as possible, you want to structure your code to send all
the heavy lifting to Google, and keep processing off of your local
resources.

In other words, your work in the Code Editor is making a description of
a computation. All ee objects are just placeholders for server-side
objects---their actual value does not exist locally on your computer. To
see or use the actual value, it has to be evaluated by the server. If
you print an Earth Engine object, it calls getInfo to evaluate and
return the value. In contrast, you can also work with JavaScript/Python
lists or numbers locally, and do basic JavaScript/Python things to them,
like add numbers together or loop over items. These are client-side
objects. Whenever you bring a server-side object into your local
environment, there's a computational cost.

Table F6.2.1 describes some nuts and bolts about Earth Engine and their
implications. Table F6.2.2 provides some of the existing limits on
individual tasks.

Table F6.2.1 Characterics of Google Earth Engine and implications for
running large jobs

Earth Engine characteristics

Implications

A parallel, distributed system

Occasionally, doing the exact same thing in two different orders can
result in different processing distributions, impacting the ability to
complete the task within system limits.

Most processing is done per tile (generally a square that is 256 x 256
pixels).

Tasks that require many tiles are the most memory intensive. Some
functions have a tileScale argument that reduces tile size, allowing
processing-intensive jobs to succeed (at the cost of reduced speed).

Export mode has higher memory and time allocations than interactive
mode.

It's better to export large jobs. You can export to your Earth Engine
assets, your Google Drive, or Google Cloud Storage.

Some operations are cached temporarily.

Running the same job twice could result in different run times.
Occasionally tasks may run successfully on a second try.

Underlying infrastructure is composed of clusters of low-end servers.

There's a hard limit on data size for any individual server; large
computations need to be done in parallel using Earth Engine functions.

The image processing domain, scale, and projection are defined by the
specified output and applied backwards throughout the processing chain.

There are not many cases when you will need to manually reproject
images, and these operations are costly. Similarly, manually
``clipping'' images is typically unnecessary.

Table F6.2.2 Size limits for Earth Engine tasks

Earth Engine Component

Limits

Interactive mode

Can print up to 5000 records. Computations must finish within five
minutes.

Export mode

Jobs have no time limit as long as they continue to make reasonable
progress (defined roughly as 600 seconds per feature, two hours per
aggregation, and 10 minutes per tile). If any one tile, feature, or
aggregation takes too long, the whole job will get canceled. Any jobs
that take longer than one week to run will likely fail due to Earth
Engine's software update release cycles.

Table assets

Maximum of 100 million features, 1000 properties (columns), and 100,000
vertices for a geometry.

\hypertarget{the-importance-of-coding-best-practices}{%
\subsubsection{The Importance of Coding Best
Practices}\label{the-importance-of-coding-best-practices}}

Good code scales better than bad code. But what is good code? Generally,
for Earth Engine, good code means 1) using Earth Engine's server-side
operators; 2) avoiding multiple passes through the same image
collection; 3) avoiding unnecessary conversions; and 4) setting the
processing scale or sample numbers appropriate for your use case, i.e.,
avoid using very fine scales or large samples without reason.

We encourage readers to become familiar with the ``Coding Best
Practices'' page in the online Earth Engine User Guide. This page
provides examples for avoiding mixing client- and server-side functions,
unnecessary conversions, costly algorithms, combining reducers, and
other helpful tips. Similarly, the ``Debugging Guide--Scaling Errors''
page of the online Earth Engine User Guide covers some common problems
and solutions.

In addition, some Earth Engine functions are more efficient than others.
For example, Image.reduceRegions is more efficient than
Image.sampleRegions, because sampleRegions regenerates the geometries
under the hood. These types of best practices are trickier to enumerate
and somewhat idiosyncratic. We encourage users to learn about and make
use of the Profiler tab, which will track and display the resources used
for each operation within your script. This can help identify areas to
focus efficiency improvements. Note that the profiler itself increases
resource use, so only use it when necessary to develop a script and
remove it for production-level execution. Other ways to discover best
practices include following/posting questions to GIS StackExchange or
the Earth Engine Developer's Discussion Group, swapping code with
others, and experimentation.

\hypertarget{scaling-across-time}{%
\subsection{Scaling Across Time}\label{scaling-across-time}}

In this section we use an example of extracting climate data at features
(points or polygons) to demonstrate how to scale an operation across
many features (Sect. 1.1) and how to break up large jobs by time units
when necessary (e.g, by years; Sect. 1.2).

\hypertarget{scaling-up-with-earth-engine-operators-annual-daily-climate-data}{%
\subsubsection{1.1. Scaling Up with Earth Engine Operators: Annual Daily
Climate
Data}\label{scaling-up-with-earth-engine-operators-annual-daily-climate-data}}

Earth Engine's operators are designed to parallelize queries on the
backend without user intervention. In many cases, they are sufficient to
accomplish a scaling operation.

As an example, we will extract a daily time series of precipitation,
maximum temperature, and minimum temperature for county polygons in the
United States. We will use the GRIDMET Climate Reanalysis product
(Abatzoglou 2013), which provides daily, 4000 m resolution gridded
meteorological data from 1979 to the present across the contiguous
United States. To save time for this practicum, we will focus on the
states of Indiana, Illinois, and Iowa in the central United States,
which together include 293 counties (Fig. F6.2.1).

\begin{figure}

{\centering \includegraphics{./F6/image11.png}

}

\caption{Fig. F6.2.1 Map of study area, showing 293 county features
within the states of Iowa, Illinois, and Indiana in the United States}

\end{figure}

This example uses the ee.Image.reduceRegions operator, which extracts
statistics from an Image for each Feature (point or polygon) in a
FeatureCollection. We will map the reduceRegions operator over each
daily image in an ImageCollection, thus providing us with the daily
climate information for each county of interest.

Note that although our example uses a climate ImageCollection, this
approach transfers to any ImageCollection, including satellite imagery,
as well as image collections that you have already processed, such as
cloud masking (Chap. F4.3) or time series aggregation (Chap. F4.2).

First, define the FeatureCollection, ImageCollection, and time period:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load county dataset.  }
\CommentTok{// Filter counties in Indiana, Illinois, and Iowa by state FIPS code.  }
\CommentTok{// Select only the unique ID column for simplicity.  }
\KeywordTok{var}\NormalTok{ countiesAll }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}TIGER/2018/Counties\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ states }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}17\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}18\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}19\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ uniqueID }\OperatorTok{=} \StringTok{\textquotesingle{}GEOID\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ featColl }\OperatorTok{=}\NormalTok{ countiesAll}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{inList}\NormalTok{(}\StringTok{\textquotesingle{}STATEFP\textquotesingle{}}\OperatorTok{,}\NormalTok{ states))  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(uniqueID)}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(featColl}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(featColl}\OperatorTok{.}\FunctionTok{limit}\NormalTok{(}\DecValTok{1}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Visualize target features (create Figure F6.2.1).  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(featColl}\OperatorTok{,} \DecValTok{5}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(featColl)}\OperatorTok{;}  
  
\CommentTok{// specify years of interest  }
\KeywordTok{var}\NormalTok{ startYear }\OperatorTok{=} \DecValTok{2020}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endYear }\OperatorTok{=} \DecValTok{2020}\OperatorTok{;}  
  
\CommentTok{// climate dataset info  }
\KeywordTok{var}\NormalTok{ imageCollectionName }\OperatorTok{=} \StringTok{\textquotesingle{}IDAHO\_EPSCOR/GRIDMET\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ bandsWanted }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}pr\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}tmmn\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}tmmx\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ scale }\OperatorTok{=} \DecValTok{4000}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Printing the size of the FeatureCollection indicates that there are 293
counties in our subset. Since we want to pull a daily time series for
one year, our final dataset will have 106,945 rows---one for each
county-day.

Note that from our county FeatureCollection, we select only the GEOID
column, which represents a unique identifier for each record in this
dataset. We do this here to simplify print outputs; we could also
specify which properties to include in the export function (see below).

Next, load and filter the climate data. Note we adjust the end date to
January 1 of the following year, rather than December 31 of the
specified year, since the filterDate function has an inclusive start
date argument and an exclusive end date argument; without this
modification the output would lack data for December 31.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load and format climate data.  }
\KeywordTok{var}\NormalTok{ startDate }\OperatorTok{=}\NormalTok{ startYear }\OperatorTok{+} \StringTok{\textquotesingle{}{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ endYear\_adj }\OperatorTok{=}\NormalTok{ endYear }\OperatorTok{+} \DecValTok{1}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=}\NormalTok{ endYear\_adj }\OperatorTok{+} \StringTok{\textquotesingle{}{-}01{-}01\textquotesingle{}}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ imageCollection }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(imageCollectionName)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(bandsWanted)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(featColl)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now get the mean value for each climate attribute within each county
feature. Here, we map the ee.Image.reduceRegions call over the
ImageCollection, specifying an ee.Reducer.mean reducer. The reducer will
apply to each band in the image, and it returns the FeatureCollection
with new properties. We also add a `date\_ymd' time property extracted
from the image to correctly associate daily values with their date.
Finally, we flatten the output to reform a single FeatureCollection with
one feature per county-day.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// get values at features  }
\KeywordTok{var}\NormalTok{ sampledFeatures }\OperatorTok{=}\NormalTok{ imageCollection}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{   }\ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{reduceRegions}\NormalTok{(\{  }
           \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ featColl}\OperatorTok{,}  
           \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,}  
           \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ scale  }
\NormalTok{       \})}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{notNull}\NormalTok{(  }
\NormalTok{       bandsWanted)) }\CommentTok{// drop rows with no data       .map(function(f) \{ // add date property           var time\_start = image.get(               \textquotesingle{}system:time\_start\textquotesingle{});           var dte = ee.Date(time\_start).format(               \textquotesingle{}YYYYMMdd\textquotesingle{});           return f.set(\textquotesingle{}date\_ymd\textquotesingle{}, dte);  }
\NormalTok{       \})}\OperatorTok{;}  
\NormalTok{\})}\OperatorTok{.}\FunctionTok{flatten}\NormalTok{()}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(sampledFeatures}\OperatorTok{.}\FunctionTok{limit}\NormalTok{(}\DecValTok{1}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Note that we include a filter to remove feature-day rows that lacked
data. While this is less common when using gridded climate products,
missing data can be common when reducing satellite images. This is
because satellite collections come in scene tiles, and each image tile
likely does not overlap all of our features unless it has first been
aggregated temporally. It can also occur if a cloud mask has been
applied to an image prior to the reduction. By filtering out null
values, we can reduce empty rows.

Now explore the result. If we simply print(sampledFeatures) we get our
first error message: ``User memory limit exceeded.'' This is because
we've created a FeatureCollection that exceeds the size limits set for
interactive mode. How many are there? We could try
print(sampledFeatures.size()), but due to the larger size, we receive a
``Computation timed out'' message---it's unable to tell us. Of course,
we know that we expect 293 counties x 365 days = 106,945 features. We
can, however, check that our reducer has worked as expected by asking
Earth Engine for just one feature: print(sampledFeatures.limit(1)).

\begin{figure}

{\centering \includegraphics{./F6/image75.png}

}

\caption{Fig. F6.2.2 Screenshot of the print output for one feature
after the reduceRegions call}

\end{figure}

Here, we can see the precipitation, minimum temperature, and maximum
temperature for the county with GEOID = 17121 on January 1, 2020 (Fig.
F6.2.2; note temperature is in Kelvin units).

Next, export the full FeatureCollection as a CSV to a folder in your
Google Drive. Specify the names of properties to include. Build part of
the filename dynamically based on arguments used for year and data
scale, so we don't need to manually modify the filenames.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// export info  }
\KeywordTok{var}\NormalTok{ exportFolder }\OperatorTok{=} \StringTok{\textquotesingle{}GEE\_scalingUp\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ filename }\OperatorTok{=} \StringTok{\textquotesingle{}Gridmet\_counties\_IN\_IL\_IA\_\textquotesingle{}} \OperatorTok{+}\NormalTok{ scale }\OperatorTok{+} \StringTok{\textquotesingle{}m\_\textquotesingle{}} \OperatorTok{+}  
\NormalTok{   startYear }\OperatorTok{+} \StringTok{\textquotesingle{}{-}\textquotesingle{}} \OperatorTok{+}\NormalTok{ endYear}\OperatorTok{;}\CommentTok{// prepare export: specify properties/columns to include  }
\KeywordTok{var}\NormalTok{ columnsWanted }\OperatorTok{=}\NormalTok{ [uniqueID]}\OperatorTok{.}\FunctionTok{concat}\NormalTok{([}\StringTok{\textquotesingle{}date\_ymd\textquotesingle{}}\NormalTok{]}\OperatorTok{,}\NormalTok{ bandsWanted)}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(columnsWanted)}\OperatorTok{;}  
  
\NormalTok{Export}\OperatorTok{.}\AttributeTok{table}\OperatorTok{.}\FunctionTok{toDrive}\NormalTok{(\{  }
   \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ sampledFeatures}\OperatorTok{,}  
   \DataTypeTok{description}\OperatorTok{:}\NormalTok{ filename}\OperatorTok{,}  
   \DataTypeTok{folder}\OperatorTok{:}\NormalTok{ exportFolder}\OperatorTok{,}  
   \DataTypeTok{fileFormat}\OperatorTok{:} \StringTok{\textquotesingle{}CSV\textquotesingle{}}\OperatorTok{,}  
   \DataTypeTok{selectors}\OperatorTok{:}\NormalTok{ columnsWanted  }
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F62a. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

On our first export, this job took about eight minutes to complete,
producing a dataset 6.8 MB in size. The data is ready for downstream use
but may need formatting to suit the user's goals. You can see what the
exported CSV looks like in Fig. F6.2.3.

\begin{figure}

{\centering \includegraphics{./F6/image41.png}

}

\caption{Fig. F6.2.3 Top six rows of the exported CSV viewed in
Microsoft Excel and sorted by county GEOID}

\end{figure}

Using the Selectors Argument

There are two excellent reasons to use the selectors argument in your
Export.table.toDrive call. First, if the argument is not specified,
Earth Engine will generate the column names for the exported CSV from
the first feature in your FeatureCollection. If that feature is missing
properties, those properties will be dropped from the export for all
features.

Perhaps even more important if you are seeking to scale up an analysis,
including unnecessary columns can greatly increase file size and even
processing time. For example, Earth Engine includes a .geo field that
contains a GeoJSON description of each spatial feature. For non-simple
geometries, the field can be quite large, as it lists coordinates for
each polygon vertex. For many purposes, it's not necessary to include
this information for each daily record (here, 365 daily rows per
feature).

For example, when we ran the same job as above but did not use the
selectors argument, the output dataset was 5.7 GB (versus 6.8 MB!) and
the runtime was slower. This is a cumbersomely large file, with no real
benefit. We generally recommend dropping the .geo column and other
unnecessary properties. To retain spatial information, a unique
identifier for each feature can be used for downstream joins with the
spatial data or other properties. If working with point data, latitude
and longitude columns can be added prior to export to maintain easily
accessible geographic information, although the .geo column for point
data is far smaller than for irregularly shaped polygon features.

\hypertarget{scaling-across-time-by-batching-get-20-years-of-daily-climate-data}{%
\subsubsection{1.2. Scaling Across Time by Batching: Get 20 Years of
Daily Climate
Data}\label{scaling-across-time-by-batching-get-20-years-of-daily-climate-data}}

Above, we extracted one year of daily data for our 293 counties. Let's
say we want to do the same thing, but for 2001--2020. We have already
written our script to flexibly specify years, so it's fairly adaptable
to this new use case:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// specify years of interest  }
\KeywordTok{var}\NormalTok{ startYear }\OperatorTok{=} \DecValTok{2020}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endYear }\OperatorTok{=} \DecValTok{2020}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

If we only wanted a few years for a small number of features, we could
just modify the startYear or endYear and proceed. Indeed, our current
example is modest in size and number of features, and we were able to
run 2001--2020 in one export job that took about two hours, with an
output file size of 299 MB. However, with larger feature collections, or
hourly data, we will again start to bump up against Earth Engine's
limits. Generally, jobs of this sort do not fail quickly---exports are
allowed to run as long as they continue making progress (see Table
F6.2.2). It's not uncommon, however, for a large job to take well over
24 hours to run, or even to fail after more than 24 hours of run time,
as it accumulates too many records or a single aggregation fails. For
users, this can be frustrating.

We generally find it simpler to run several small jobs rather than one
large job. Outputs can then be combined in external software. This
avoids any frustration with long-running jobs or delayed failures, and
it allows parts of the task to be run simultaneously. Earth Engine
generally executes from 2--20 jobs per user at a time, depending on
overall user load (although 20 is rare). As a counterpart, there is some
overhead for generating separate jobs.

Important note: When running a batch of jobs, it may be tempting to use
multiple accounts to execute subsets of your batch and thus get your
shared results faster. However, doing so is a direct violation of the
Earth Engine terms of service and can result in your account(s) being
terminated.

For-Loops: They Are Sometimes OK

Batching jobs in time is a great way to break up a task into smaller
units. Other options include batching jobs by spatial regions defined by
polygons (see Sect. 2), or for computationally heavy tasks, batching by
both space and time.

Because Export functions are client-side functions, however, you can't
create an export within an Earth Engine map command. Instead, we need to
loop over the variable that will define our batches and create a set of
export tasks.

But wait! Aren't we supposed to avoid for-loops at all costs? Yes,
within a computational chain. Here, we are using a loop to send multiple
computational chains to the server.

First, we will start with the same script as in Sect. 1.1, but we will
modify the start year. We will also modify the desired output filename
to be a generic base filename, to which we will append the year for each
task within the loop (in the next step).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Load county dataset.  }
\KeywordTok{var}\NormalTok{ countiesAll }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}TIGER/2018/Counties\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ states }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}17\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}18\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}19\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ uniqueID }\OperatorTok{=} \StringTok{\textquotesingle{}GEOID\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ featColl }\OperatorTok{=}\NormalTok{ countiesAll}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{inList}\NormalTok{(}\StringTok{\textquotesingle{}STATEFP\textquotesingle{}}\OperatorTok{,}\NormalTok{ states))  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{(uniqueID)}\OperatorTok{;}  
  
\FunctionTok{print}\NormalTok{(featColl}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}  
\FunctionTok{print}\NormalTok{(featColl}\OperatorTok{.}\FunctionTok{limit}\NormalTok{(}\DecValTok{1}\NormalTok{))}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(featColl)}\OperatorTok{;}  
  
\CommentTok{// Specify years of interest.  }
\KeywordTok{var}\NormalTok{ startYear }\OperatorTok{=} \DecValTok{2001}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ endYear }\OperatorTok{=} \DecValTok{2020}\OperatorTok{;}  
  
\CommentTok{// Climate dataset info.  }
\KeywordTok{var}\NormalTok{ imageCollectionName }\OperatorTok{=} \StringTok{\textquotesingle{}IDAHO\_EPSCOR/GRIDMET\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ bandsWanted }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}pr\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}tmmn\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}tmmx\textquotesingle{}}\NormalTok{]}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ scale }\OperatorTok{=} \DecValTok{4000}\OperatorTok{;}  
  
\CommentTok{// Export info.  }
\KeywordTok{var}\NormalTok{ exportFolder }\OperatorTok{=} \StringTok{\textquotesingle{}GEE\_scalingUp\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ filenameBase }\OperatorTok{=} \StringTok{\textquotesingle{}Gridmet\_counties\_IN\_IL\_IA\_\textquotesingle{}} \OperatorTok{+}\NormalTok{ scale }\OperatorTok{+} \StringTok{\textquotesingle{}m\_\textquotesingle{}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now modify the code in Sect. 1.1 to use a looping variable, i, to
represent each year. Here, we are using standard JavaScript looping
syntax, where i will take on each value between our startYear (2001) and
our endYear (2020) for each loop through this section of code, thus
creating 20 queries to send to Earth Engine's servers.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Initiate a loop, in which the variable i takes on values of each year.  }
\ControlFlowTok{for}\NormalTok{ (}\KeywordTok{var}\NormalTok{ i }\OperatorTok{=}\NormalTok{ startYear}\OperatorTok{;}\NormalTok{ i }\OperatorTok{\textless{}=}\NormalTok{ endYear}\OperatorTok{;}\NormalTok{ i}\OperatorTok{++}\NormalTok{) \{        }\CommentTok{// for each year....  // Load climate collection for that year. var startDate = i + \textquotesingle{}{-}01{-}01\textquotesingle{};  }
  
  \KeywordTok{var}\NormalTok{ endYear\_adj }\OperatorTok{=}\NormalTok{ i }\OperatorTok{+} \DecValTok{1}\OperatorTok{;} \KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=}\NormalTok{ endYear\_adj }\OperatorTok{+} \StringTok{\textquotesingle{}{-}01{-}01\textquotesingle{}}\OperatorTok{;} \KeywordTok{var}\NormalTok{ imageCollection }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(imageCollectionName)  }
     \OperatorTok{.}\FunctionTok{select}\NormalTok{(bandsWanted)  }
     \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(featColl)  }
     \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)}\OperatorTok{;} \CommentTok{// Get values at feature collection. var sampledFeatures = imageCollection.map(function(image) \{   return image.reduceRegions(\{  }
     \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ featColl}\OperatorTok{,}  
     \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,}          
     \DataTypeTok{tileScale}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}  
     \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ scale  }
\NormalTok{   \})}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{notNull}\NormalTok{(bandsWanted))  }\CommentTok{// remove rows without data     .map(function(f) \{                  // add date property       var time\_start = image.get(\textquotesingle{}system:time\_start\textquotesingle{});       var dte = ee.Date(time\_start).format(\textquotesingle{}YYYYMMdd\textquotesingle{});       return f.set(\textquotesingle{}date\_ymd\textquotesingle{}, dte);  }
\NormalTok{   \})}\OperatorTok{;}  
\NormalTok{ \})}\OperatorTok{.}\FunctionTok{flatten}\NormalTok{()}\OperatorTok{;} \CommentTok{// Prepare export: specify properties and filename. var columnsWanted = [uniqueID].concat([\textquotesingle{}date\_ymd\textquotesingle{}], bandsWanted); var filename = filenameBase + i; Export.table.toDrive(\{  }
\NormalTok{   collection}\OperatorTok{:}\NormalTok{ sampledFeatures}\OperatorTok{,}  
\NormalTok{   description}\OperatorTok{:}\NormalTok{ filename}\OperatorTok{,}  
\NormalTok{   folder}\OperatorTok{:}\NormalTok{ exportFolder}\OperatorTok{,}  
\NormalTok{   fileFormat}\OperatorTok{:} \StringTok{\textquotesingle{}CSV\textquotesingle{}}\OperatorTok{,}  
\NormalTok{   selectors}\OperatorTok{:}\NormalTok{ columnsWanted  }
\NormalTok{ \})}\OperatorTok{;}  
   
\NormalTok{\}}

\OperatorTok{:::}\NormalTok{\{}\OperatorTok{.}\AttributeTok{callout}\OperatorTok{{-}}\NormalTok{note\}}
\NormalTok{Code Checkpoint F62b}\OperatorTok{.} \AttributeTok{The}\NormalTok{ book’s repository contains a script that shows what your code should look like at }\KeywordTok{this}\NormalTok{ point}\OperatorTok{.}
\OperatorTok{:::}
\NormalTok{When we run }\KeywordTok{this}\NormalTok{ script}\OperatorTok{,}\NormalTok{ it builds our computational query }\ControlFlowTok{for}\NormalTok{ each year}\OperatorTok{,}\NormalTok{ creating a batch }\KeywordTok{of} \DecValTok{20}\NormalTok{ individual jobs that will show up }\KeywordTok{in}\NormalTok{ the Task }\FunctionTok{pane}\NormalTok{ (Fig}\OperatorTok{.} \AttributeTok{F6}\OperatorTok{.}\FloatTok{2.4}\NormalTok{)}\OperatorTok{.} \AttributeTok{Each}\NormalTok{ task name includes the year}\OperatorTok{,}\NormalTok{ since we used our looping variable i to modify the base filename we specified}\OperatorTok{.}

\OperatorTok{!}\NormalTok{[Fig}\OperatorTok{.} \AttributeTok{F6}\OperatorTok{.}\FloatTok{2.4}\NormalTok{ Creation }\KeywordTok{of}\NormalTok{ batch tasks }\ControlFlowTok{for}\NormalTok{ each year](F6}\OperatorTok{/}\NormalTok{image63}\OperatorTok{.}\AttributeTok{png}\NormalTok{)}


\NormalTok{We now encounter a downside to creating batch tasks within the JavaScript Code Editor}\OperatorTok{:}\NormalTok{ we need to click Run to execute each job }\KeywordTok{in}\NormalTok{ turn}\OperatorTok{.} \AttributeTok{Here}\OperatorTok{,}\NormalTok{ we made }\KeywordTok{this}\NormalTok{ easier by programmatically assigning each job the filename we want}\OperatorTok{,}\NormalTok{ so we can hold the Cmd}\OperatorTok{/}\NormalTok{Ctrl key and click Run to avoid the }\ImportTok{export}\NormalTok{ task option }\BuiltInTok{window}\NormalTok{ and only need to click once per task}\OperatorTok{.} \AttributeTok{Still}\OperatorTok{,}\NormalTok{ one can imagine that at some number }\KeywordTok{of}\NormalTok{ tasks}\OperatorTok{,}\NormalTok{ one’s patience }\ControlFlowTok{for}\NormalTok{ clicking Run will be exceeded}\OperatorTok{.} \AttributeTok{We}\NormalTok{ assume that number is different }\ControlFlowTok{for}\NormalTok{ everyone}\OperatorTok{.}

\NormalTok{Note}\OperatorTok{:}\NormalTok{ If at any time you have submitted several tasks to the server but want to cancel them all}\OperatorTok{,}\NormalTok{ you can }\ControlFlowTok{do}\NormalTok{ so more easily }\ImportTok{from}\NormalTok{ the Earth Engine Task Manager that is linked at the top }\KeywordTok{of}\NormalTok{ the Task pane}\OperatorTok{.} \AttributeTok{You}\NormalTok{ can read about that task manager }\KeywordTok{in}\NormalTok{ the Earth Engine User Guide}\OperatorTok{.}

\NormalTok{In order to auto}\OperatorTok{{-}}\NormalTok{execute jobs }\KeywordTok{in}\NormalTok{ batch mode}\OperatorTok{,}\NormalTok{ we’d need to use the Python API}\OperatorTok{.} \AttributeTok{Interested}\NormalTok{ users can see the Earth Engine User Guide Python API tutorial }\ControlFlowTok{for}\NormalTok{ further details about the Python API}\OperatorTok{.} 

\NormalTok{\#\#\# Scaling Across Space via Spatial Tiling}

\NormalTok{Breaking up jobs }\KeywordTok{in}\NormalTok{ space is another key strategy }\ControlFlowTok{for}\NormalTok{ scaling operations }\KeywordTok{in}\NormalTok{ Earth Engine}\OperatorTok{.} \AttributeTok{Here}\OperatorTok{,}\NormalTok{ we will focus on making a cloud}\OperatorTok{{-}}\NormalTok{free composite }\ImportTok{from}\NormalTok{ the Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ Level }\DecValTok{2}\ErrorTok{A}\NormalTok{ Surface Reflectance product}\OperatorTok{.} \AttributeTok{The}\NormalTok{ approach is similar to that }\KeywordTok{in}\NormalTok{ Chap}\OperatorTok{.} \AttributeTok{F4}\OperatorTok{.}\DecValTok{3}\OperatorTok{,}\NormalTok{ which explores cloud}\OperatorTok{{-}}\NormalTok{free compositing}\OperatorTok{.} \AttributeTok{The}\NormalTok{ main difference is that Landsat scenes come }\ControlFlowTok{with}\NormalTok{ a reliable quality band }\ControlFlowTok{for}\NormalTok{ each scene}\OperatorTok{,}\NormalTok{ whereas the }\BuiltInTok{process} \ControlFlowTok{for}\NormalTok{ Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ is a bit more complicated and computationally }\FunctionTok{intense}\NormalTok{ (see below)}\OperatorTok{.}

\NormalTok{Our region }\KeywordTok{of}\NormalTok{ interest is the state }\KeywordTok{of}\NormalTok{ Washington }\KeywordTok{in}\NormalTok{ the United States }\ControlFlowTok{for}\NormalTok{ demonstration purposes}\OperatorTok{,}\NormalTok{ but the method will work at much larger continental scales }\ImportTok{as}\NormalTok{ well}\OperatorTok{.}

\NormalTok{Cloud Masking Approach}

\NormalTok{While we }\ControlFlowTok{do}\NormalTok{ not intend to cover the theory behind Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ cloud masking}\OperatorTok{,}\NormalTok{ we }\ControlFlowTok{do}\NormalTok{ want to include a brief description }\KeywordTok{of}\NormalTok{ the }\BuiltInTok{process}\NormalTok{ to convey the computational needs }\KeywordTok{of} \KeywordTok{this}\NormalTok{ approach}\OperatorTok{.}

\NormalTok{The Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ Level }\DecValTok{2}\ErrorTok{A}\NormalTok{ collection does not come }\ControlFlowTok{with}\NormalTok{ a robust cloud mask}\OperatorTok{.} \AttributeTok{Instead}\OperatorTok{,}\NormalTok{ we will build one }\ImportTok{from}\NormalTok{ related products that have been developed }\ControlFlowTok{for} \KeywordTok{this}\NormalTok{ purpose}\OperatorTok{.} \AttributeTok{Following}\NormalTok{ the existing Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ cloud masking tutorials }\KeywordTok{in}\NormalTok{ the Earth Engine guides}\OperatorTok{,} \KeywordTok{this}\NormalTok{ approach requires three Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ image collections}\OperatorTok{:}

\OperatorTok{*}\NormalTok{   The Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ Level }\DecValTok{2}\ErrorTok{A}\NormalTok{ Surface Reflectance product}\OperatorTok{.} \AttributeTok{This}\NormalTok{ is the dataset we want to use to build our final composite}\OperatorTok{.}
\OperatorTok{*}\NormalTok{   The Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ Cloud Probability Dataset}\OperatorTok{,}\NormalTok{ an ImageCollection that contains cloud probabilities }\ControlFlowTok{for}\NormalTok{ each Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ scene}\OperatorTok{.}
\OperatorTok{*}\NormalTok{   The Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ Level }\DecValTok{1}\ErrorTok{C}\NormalTok{ top}\OperatorTok{{-}}\KeywordTok{of}\OperatorTok{{-}}\NormalTok{atmosphere product}\OperatorTok{.} \AttributeTok{This}\NormalTok{ collection is needed to run the Cloud Displacement }\BuiltInTok{Index}\NormalTok{ to identify cloud shadows}\OperatorTok{,}\NormalTok{ which is calculated using ee}\OperatorTok{.}\AttributeTok{Algorithms}\OperatorTok{.}\AttributeTok{Sentinel2}\OperatorTok{.}\FunctionTok{CDI}\NormalTok{ (see Frantz et al}\OperatorTok{.} \DecValTok{2018} \ControlFlowTok{for}\NormalTok{ algorithm description)}\OperatorTok{.}

\NormalTok{These three image collections all contain }\DecValTok{10}\NormalTok{ m resolution data }\ControlFlowTok{for}\NormalTok{ every Sentinel}\OperatorTok{{-}}\DecValTok{2}\NormalTok{ scene}\OperatorTok{.} \AttributeTok{We}\NormalTok{ will join them based on their }\StringTok{\textquotesingle{}system:index\textquotesingle{}}\NormalTok{ property so we can relate each Level }\DecValTok{2}\ErrorTok{A}\NormalTok{ scene }\ControlFlowTok{with}\NormalTok{ the corresponding cloud probability and cloud displacement index}\OperatorTok{.} \AttributeTok{Furthermore}\OperatorTok{,}\NormalTok{ there are two ee}\OperatorTok{.}\AttributeTok{Image}\OperatorTok{.}\AttributeTok{projection}\NormalTok{ steps to control the scale when calculating clouds and their shadows}\OperatorTok{.}

\NormalTok{To sum up}\OperatorTok{,}\NormalTok{ the cloud masking approach is computationally costly}\OperatorTok{,}\NormalTok{ thus requiring some thought when applying it at scale}\OperatorTok{.}

\NormalTok{\#\#\#\# }\FloatTok{2.1}\OperatorTok{.} \AttributeTok{Generate}\NormalTok{ a Cloud}\OperatorTok{{-}}\NormalTok{Free Satellite Composite}\OperatorTok{:}\NormalTok{ Limits to On}\OperatorTok{{-}}\NormalTok{the}\OperatorTok{{-}}\NormalTok{Fly Computing}

\NormalTok{Note}\OperatorTok{:}\NormalTok{ Our focus here is on code structure }\ControlFlowTok{for}\NormalTok{ implementing spatial tiling}\OperatorTok{.} \AttributeTok{Below}\OperatorTok{,}\NormalTok{ we }\ImportTok{import}\NormalTok{ existing tested functions }\ControlFlowTok{for}\NormalTok{ cloud masking using the require command}\OperatorTok{.}

\NormalTok{First}\OperatorTok{,}\NormalTok{ define our region and time }\KeywordTok{of}\NormalTok{ interest}\OperatorTok{;}\NormalTok{ then}\OperatorTok{,}\NormalTok{ load the module containing the cloud functions}\OperatorTok{.}

\CommentTok{// Set the Region of Interest:Seattle, Washington, United States  }
\KeywordTok{var}\NormalTok{ roi }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{122.33524518034544}\OperatorTok{,} \FloatTok{47.61356183942883}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Dates over which to create a median composite.  }
\KeywordTok{var}\NormalTok{ start }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(}\StringTok{\textquotesingle{}2019{-}03{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ end }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(}\StringTok{\textquotesingle{}2019{-}09{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Specify module with cloud mask functions.  }
\KeywordTok{var}\NormalTok{ s2mask\_tools }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(   }\StringTok{\textquotesingle{}projects/gee{-}edu/book:Part F {-} Fundamentals/F6 {-} Advanced Topics/F6.2 Scaling Up/modules/s2cloudmask.js\textquotesingle{}}  
\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, load and filter our three Sentinel-2 image collections.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Sentinel{-}2 surface reflectance data for the composite.  }
\KeywordTok{var}\NormalTok{ s2Sr }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\_SR\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(roi)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B5\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Sentinel{-}2 Level 1C data (top{-}of{-}atmosphere).    }
\CommentTok{// Bands B7, B8, B8A and B10 needed for CDI and the cloud mask function.  }
\KeywordTok{var}\NormalTok{ s2 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(roi)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end)  }
   \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}B7\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B8A\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B10\textquotesingle{}}\NormalTok{])}\OperatorTok{;}  
  
\CommentTok{// Cloud probability dataset {-} used in cloud mask function  }
\KeywordTok{var}\NormalTok{ s2c }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\_CLOUD\_PROBABILITY\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end)  }
   \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(roi)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now apply the cloud mask:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Join the cloud probability dataset to surface reflectance.  }
\KeywordTok{var}\NormalTok{ withCloudProbability }\OperatorTok{=}\NormalTok{ s2mask\_tools}\OperatorTok{.}\FunctionTok{indexJoin}\NormalTok{(s2Sr}\OperatorTok{,}\NormalTok{ s2c}\OperatorTok{,}   \StringTok{\textquotesingle{}cloud\_probability\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Join the L1C data to get the bands needed for CDI.  }
\KeywordTok{var}\NormalTok{ withS2L1C }\OperatorTok{=}\NormalTok{ s2mask\_tools}\OperatorTok{.}\FunctionTok{indexJoin}\NormalTok{(withCloudProbability}\OperatorTok{,}\NormalTok{ s2}\OperatorTok{,}   \StringTok{\textquotesingle{}l1c\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Map the cloud masking function over the joined collection.  }
\CommentTok{// Cast output to ImageCollection  }
\KeywordTok{var}\NormalTok{ masked }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(withS2L1C}\OperatorTok{.}\FunctionTok{map}\NormalTok{(s2mask\_tools  }
\OperatorTok{.}\AttributeTok{maskImage}\NormalTok{))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, generate and visualize the median composite:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Take the median, specifying a tileScale to avoid memory errors.  }
\KeywordTok{var}\NormalTok{ median }\OperatorTok{=}\NormalTok{ masked}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{median}\NormalTok{()}\OperatorTok{,} \DecValTok{8}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Display the results.  }
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(roi}\OperatorTok{,} \DecValTok{12}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(roi)}\OperatorTok{;}  
  
\KeywordTok{var}\NormalTok{ viz }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\_median\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\_median\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\_median\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{3000}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(median}\OperatorTok{,}\NormalTok{ viz}\OperatorTok{,} \StringTok{\textquotesingle{}median\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F62c. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

After about 1--3 minutes, Earth Engine returns our composite to us on
the fly (Fig. F6.2.5). Note that panning and zooming to a new area
requires that Earth Engine must again issue the compositing request to
calculate the image for new areas. Given the delay, this isn't a very
satisfying way to explore our composite.

\includegraphics{./F6/image49.png}\includegraphics{./F6/image43.png}

Next, expand our view (set zoom to 9) to exceed the limits of on-the-fly
computation (Fig. F6.2.6).

Map.centerObject(roi, 9);\\
Map.addLayer(roi);\\
Map.addLayer(median, viz, `median');

\begin{figure}

{\centering \includegraphics{./F6/image42.png}

}

\caption{Fig. F6.2.6 Error message for exceeding memory limits in
interactive mode}

\end{figure}

As you can see, this is an excellent candidate for an export task rather
than running in ``on-the-fly'' interactive mode, as above.

\hypertarget{generate-a-regional-composite-through-spatial-tiling}{%
\subsubsection{2.2. Generate a Regional Composite Through Spatial
Tiling}\label{generate-a-regional-composite-through-spatial-tiling}}

Our goal is to apply the cloud masking method in Sect. 2.1 to the state
of Washington, United States. In our testing, we successfully exported
one Sentinel-2 composite for this area in about nine hours, but for this
tutorial, let's presume we need to split the task up to be successful.

Essentially, we want to split our region of interest up into a regular
grid. For each grid, we will export a composite image into a new
ImageCollection asset. We can then load and mosaic our composite for use
in downstream scripts (see below).

First, generate a spatial polygon grid (FeatureCollection) of desired
size over your region of interest (see Fig. F6.2.7):

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Specify helper functions.  }
\KeywordTok{var}\NormalTok{ s2mask\_tools }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(   }\StringTok{\textquotesingle{}projects/gee{-}edu/book:Part F {-} Fundamentals/F6 {-} Advanced Topics/F6.2 Scaling Up/modules/s2cloudmask.js\textquotesingle{}}  
\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Set the Region of Interest: Washington, USA.  }
\KeywordTok{var}\NormalTok{ roi }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{\textquotesingle{}TIGER/2018/States\textquotesingle{}}\NormalTok{)  }
   \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{equals}\NormalTok{(}\StringTok{\textquotesingle{}NAME\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}Washington\textquotesingle{}}\NormalTok{))}\OperatorTok{;}  
  
\CommentTok{// Specify grid size in projection, x and y units (based on projection).  }
\KeywordTok{var}\NormalTok{ projection }\OperatorTok{=} \StringTok{\textquotesingle{}EPSG:4326\textquotesingle{}}\OperatorTok{;} \CommentTok{// WGS84 lat lon  }
\KeywordTok{var}\NormalTok{ dx }\OperatorTok{=} \FloatTok{2.5}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ dy }\OperatorTok{=} \FloatTok{1.5}\OperatorTok{;}  
  
\CommentTok{// Dates over which to create a median composite.  }
\KeywordTok{var}\NormalTok{ start }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(}\StringTok{\textquotesingle{}2019{-}03{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ end }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(}\StringTok{\textquotesingle{}2019{-}09{-}01\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
  
\CommentTok{// Make grid and visualize.  }
\KeywordTok{var}\NormalTok{ proj }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Projection}\NormalTok{(projection)}\OperatorTok{.}\FunctionTok{scale}\NormalTok{(dx}\OperatorTok{,}\NormalTok{ dy)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ grid }\OperatorTok{=}\NormalTok{ roi}\OperatorTok{.}\FunctionTok{geometry}\NormalTok{()}\OperatorTok{.}\FunctionTok{coveringGrid}\NormalTok{(proj)}\OperatorTok{;}  
  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(roi}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}roi\textquotesingle{}}\NormalTok{)}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(grid}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,} \StringTok{\textquotesingle{}grid\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./F6/image12.png}

}

\caption{Fig. F6.2.7 Visualization of the regular spatial grid generated
for use in spatial batch processing}

\end{figure}

Next, create a new, empty ImageCollection asset to use as our export
destination (Assets \textgreater{} New \textgreater{} Image Collection;
Fig. F6.2.8). Name the image collection `S2\_composite\_WA' and specify
the asset location in your user folder (e.g.,
``path/to/your/asset/s2\_composite\_WA'').

\begin{figure}

{\centering \includegraphics{./F6/image15.png}

}

\caption{Fig. F6.2.8 The ``create new image collection asset'' menu in
the Code Editor}

\end{figure}

Specify the ImageCollection to export to, along with a base name for
each image (the tile number will be appended in the batch export).

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Export info.  }
\KeywordTok{var}\NormalTok{ assetCollection }\OperatorTok{=} \StringTok{\textquotesingle{}path/to/your/asset/s2\_composite\_WA\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ imageBaseName }\OperatorTok{=} \StringTok{\textquotesingle{}S2\_median\_\textquotesingle{}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Extract grid numbers to use as looping variables. Note there is one
getInfo call here, which should be used sparingly and never within a
for-loop if you can help it. We use it to bring the number of grid cells
we've generated onto the client-side to set up the for-loop over grids.
Note that if your grid has too many elements, you may need a different
strategy.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Get a list based on grid number.  }
\KeywordTok{var}\NormalTok{ gridSize }\OperatorTok{=}\NormalTok{ grid}\OperatorTok{.}\FunctionTok{size}\NormalTok{()}\OperatorTok{.}\FunctionTok{getInfo}\NormalTok{()}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ gridList }\OperatorTok{=}\NormalTok{ grid}\OperatorTok{.}\FunctionTok{toList}\NormalTok{(gridSize)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Batch generate a composite image task export for each grid via looping:

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// In each grid cell, export a composite  }
\ControlFlowTok{for}\NormalTok{ (}\KeywordTok{var}\NormalTok{ i }\OperatorTok{=} \DecValTok{0}\OperatorTok{;}\NormalTok{ i }\OperatorTok{\textless{}}\NormalTok{ gridSize}\OperatorTok{;}\NormalTok{ i}\OperatorTok{++}\NormalTok{) \{   }\CommentTok{// Extract grid polygon and filter S2 datasets for this region.   var gridCell = ee.Feature(gridList.get(i)).geometry();   var s2Sr = ee.ImageCollection(\textquotesingle{}COPERNICUS/S2\_SR\textquotesingle{})  }
       \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end)  }
       \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(gridCell)  }
       \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B5\textquotesingle{}}\NormalTok{])}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ s2 }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\textquotesingle{}}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end)  }
       \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(gridCell)  }
       \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}B7\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B8A\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B10\textquotesingle{}}\NormalTok{])}\OperatorTok{;}   \KeywordTok{var}\NormalTok{ s2c }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\_CLOUD\_PROBABILITY\textquotesingle{}}\NormalTok{)  }
       \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end)  }
       \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(gridCell)}\OperatorTok{;}   \CommentTok{// Apply the cloud mask.   var withCloudProbability = s2mask\_tools.indexJoin(s2Sr, s2c,       \textquotesingle{}cloud\_probability\textquotesingle{});   var withS2L1C = s2mask\_tools.indexJoin(withCloudProbability, s2,       \textquotesingle{}l1c\textquotesingle{});   var masked = ee.ImageCollection(withS2L1C.map(s2mask\_tools  }
       \OperatorTok{.}\AttributeTok{maskImage}\NormalTok{))}\OperatorTok{;}   \CommentTok{// Generate a median composite and export.   var median = masked.reduce(ee.Reducer.median(), 8);   // Export.   var imagename = imageBaseName + \textquotesingle{}tile\textquotesingle{} + i;   Export.image.toAsset(\{  }
       \DataTypeTok{image}\OperatorTok{:}\NormalTok{ median}\OperatorTok{,}  
       \DataTypeTok{description}\OperatorTok{:}\NormalTok{ imagename}\OperatorTok{,}  
       \DataTypeTok{assetId}\OperatorTok{:}\NormalTok{ assetCollection }\OperatorTok{+} \StringTok{\textquotesingle{}/\textquotesingle{}} \OperatorTok{+}\NormalTok{ imagename}\OperatorTok{,}  
       \DataTypeTok{scale}\OperatorTok{:} \DecValTok{10}\OperatorTok{,}  
       \DataTypeTok{region}\OperatorTok{:}\NormalTok{ gridCell}\OperatorTok{,}  
       \DataTypeTok{maxPixels}\OperatorTok{:} \FloatTok{1e13}\NormalTok{   \})}\OperatorTok{;}  
\NormalTok{\}}

\OperatorTok{:::}\NormalTok{\{}\OperatorTok{.}\AttributeTok{callout}\OperatorTok{{-}}\NormalTok{note\}}
\NormalTok{Code Checkpoint F62d}\OperatorTok{.} \AttributeTok{The}\NormalTok{ book’s repository contains a script that shows what your code should look like at }\KeywordTok{this}\NormalTok{ point}\OperatorTok{.}
\OperatorTok{:::}
\NormalTok{Similar to Sect}\OperatorTok{.} \FloatTok{1.2}\OperatorTok{,}\NormalTok{ we now have a list }\KeywordTok{of}\NormalTok{ tasks to execute}\OperatorTok{.} \AttributeTok{We}\NormalTok{ can hold the Cmd}\OperatorTok{/}\NormalTok{Ctrl key and click Run to execute each }\FunctionTok{task}\NormalTok{ (Fig}\OperatorTok{.} \AttributeTok{F6}\OperatorTok{.}\FloatTok{2.9}\NormalTok{)}\OperatorTok{.} \AttributeTok{Again}\OperatorTok{,}\NormalTok{ users }\ControlFlowTok{with}\NormalTok{ applications requiring large batches may want to explore the Earth Engine Python API}\OperatorTok{,}\NormalTok{ which is well}\OperatorTok{{-}}\NormalTok{suited to batching work}\OperatorTok{.} \AttributeTok{The}\NormalTok{ output ImageCollection is }\FloatTok{35.3}\NormalTok{ GB}\OperatorTok{,}\NormalTok{ so you may not want to execute }\FunctionTok{all}\NormalTok{ (or any) }\KeywordTok{of}\NormalTok{ these tasks but can access our pre}\OperatorTok{{-}}\NormalTok{generated image}\OperatorTok{,} \ImportTok{as}\NormalTok{ discussed below}\OperatorTok{.}

\OperatorTok{!}\NormalTok{[Fig}\OperatorTok{.} \AttributeTok{F6}\OperatorTok{.}\FloatTok{2.9}\NormalTok{ Spatial batch tasks have been generated and are ready to run](F6}\OperatorTok{/}\NormalTok{image22}\OperatorTok{.}\AttributeTok{png}\NormalTok{)}


\NormalTok{In addition to being necessary }\ControlFlowTok{for}\NormalTok{ very large regions}\OperatorTok{,}\NormalTok{ batch processing can speed things up }\ControlFlowTok{for}\NormalTok{ moderate scales}\OperatorTok{.} \AttributeTok{In}\NormalTok{ our tests}\OperatorTok{,}\NormalTok{ tiles averaged about one hour to complete}\OperatorTok{.} \AttributeTok{Because}\NormalTok{ three jobs }\KeywordTok{in}\NormalTok{ our queue were running simultaneously}\OperatorTok{,}\NormalTok{ we covered the full state }\KeywordTok{of}\NormalTok{ Washington }\KeywordTok{in}\NormalTok{ about four }\FunctionTok{hours}\NormalTok{ (compared to about nine hours when tested }\ControlFlowTok{for}\NormalTok{ the full state }\KeywordTok{of}\NormalTok{ Washington at once)}\OperatorTok{.} \AttributeTok{Users}\NormalTok{ should note}\OperatorTok{,}\NormalTok{ however}\OperatorTok{,}\NormalTok{ that there is also an overhead to spinning up each batch task}\OperatorTok{.} \AttributeTok{Finding}\NormalTok{ the balance between task size and task number is a challenge }\ControlFlowTok{for}\NormalTok{ most Earth Engine users that becomes easier }\ControlFlowTok{with}\NormalTok{ experience}\OperatorTok{.}

\NormalTok{In a }\KeywordTok{new}\NormalTok{ script}\OperatorTok{,}\NormalTok{ load the exported ImageCollection and mosaic }\ControlFlowTok{for}\NormalTok{ use}\OperatorTok{.}

\CommentTok{// load image collection and mosaic into single image  }
\KeywordTok{var}\NormalTok{ assetCollection }\OperatorTok{=} \StringTok{\textquotesingle{}projects/gee{-}book/assets/F6{-}2/s2\_composite\_WA\textquotesingle{}}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ composite }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(assetCollection)}\OperatorTok{.}\FunctionTok{mosaic}\NormalTok{()}\OperatorTok{;}  
  
\CommentTok{// Display the results  }
\KeywordTok{var}\NormalTok{ geometry }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{([}\OperatorTok{{-}}\FloatTok{120.5873563817392}\OperatorTok{,}   \FloatTok{47.39035206888694}  
\NormalTok{])}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(geometry}\OperatorTok{,} \DecValTok{6}\NormalTok{)}\OperatorTok{;}  
\KeywordTok{var}\NormalTok{ vizParams }\OperatorTok{=}\NormalTok{ \{  }
   \DataTypeTok{bands}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}B4\_median\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\_median\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\_median\textquotesingle{}}\NormalTok{]}\OperatorTok{,}  
   \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,}  
   \DataTypeTok{max}\OperatorTok{:} \DecValTok{3000}  
\NormalTok{\}}\OperatorTok{;}  
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(composite}\OperatorTok{,}\NormalTok{ vizParams}\OperatorTok{,} \StringTok{\textquotesingle{}median\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F62e. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

\begin{figure}

{\centering \includegraphics{./F6/image16.png}

}

\caption{Fig. F6.2.10 Sentinel-2 composite covering the state of
Washington, loaded from asset. The remaining white colors are
snow-capped mountains, not clouds.}

\end{figure}

Note the ease, speed, and joy of panning and zooming to explore the
pre-computed composite asset (Fig. F6.2.10) compared to the on-the-fly
version discussed in Sect. 2.1.

\hypertarget{multistep-workflows-and-intermediate-assets}{%
\subsection{Multistep Workflows and Intermediate
Assets}\label{multistep-workflows-and-intermediate-assets}}

Often, our goals require several processing steps that cannot be
completed within one Earth Engine computational chain. In these cases,
the best strategy becomes breaking down tasks into individual pieces
that are created, stored in assets, and used across several scripts.
Each sequential script creates an intermediate output, and this
intermediate output becomes the input to the next script.

As an example, consider the land use classification task of identifying
irrigated agricultural lands. This type of classification can benefit
from several types of evidence, including satellite composites,
aggregated weather information, soil information, and/or crop type
locations. Individual steps for this type of work might include:

\begin{itemize}
\tightlist
\item
  Generating satellite composites of annual or monthly vegetation
  indices
\item
  Processing climate data into monthly or seasonal values
\item
  Generating random point locations from a ground truth layer for use as
  a feature training dataset and accuracy validation, and extracting
  composite and weather values at these features
\item
  Training a classifier and applying it, possibly across multiple years;
  researchers will often implement multiple classifiers and compare the
  performance of different methods
\item
  Implementing post-classification cleaning steps, such as removing
  ``speckle''
\item
  Evaluating accuracy at ground truth validation points, and against
  government statistics using total area per administrative boundary
\item
  Exporting your work as spatial layers, visualizations, or other
  formats
\end{itemize}

Multipart workflows can become unwieldy to manage, particularly if there
are multiple collaborators or the project has a long timeline; it can be
difficult to remember why each script was developed and where it fits in
the overall workflow.

Here, we provide tips for managing multipart workflows. These are
somewhat opinionated and based largely on concepts from ``Good Enough
Practices in Scientific Computing'' (Wilson et al.~2017). Ultimately,
your personal workflow practices will be a combination of what works for
you, what works for your larger team and organization, and, hopefully,
what works for good documentation and reproducibility.

Tip 1. Create a repository for each project

The repository can be considered the fundamental project unit. In Earth
Engine, sharing permissions are set for each individual repository, so
this allows you to share a specific project with others (see Chap.
F6.1).

By default, Earth Engine saves new scripts in a ``default'' repository
specific for each user (users//default). You can create new repositories
on the Scripts tab of the Code Editor (Fig. F6.2.11).

\begin{figure}

{\centering \includegraphics{./F6/image20.png}

}

\caption{Fig. F6.2.11 The Code Editor menu for creating new
repositories}

\end{figure}

To adjust permissions for each repository, click on the Gear icon (Fig.
F6.2.12):

\begin{figure}

{\centering \includegraphics{./F6/image67.png}

}

\caption{Fig. F6.2.12 Access the sharing and permissions menu for each
repository by clicking the Gear icon}

\end{figure}

For users familiar with version control, Earth Engine uses a git-based
script manager, so each repository can also be managed, edited, and/or
synced with your local copy or collaborative spaces like GitHub.

Tip 2. Make a separate script for each step, and make script file names
informative and self-sorting

Descriptive, self-sorting filenames are an excellent ``good enough'' way
to keep your projects organized. We recommend starting script names with
zero-padded numeric values to take advantage of default ordering.
Because we are generating assets in early scripts that are used in later
scripts, it's important to preserve the order of your workflow. The name
should also include short descriptions of what the script does (Fig.
F6.2.13).

\begin{figure}

{\centering \includegraphics{./F6/image62.png}

}

\caption{Fig. F6.2.13 An example project repository with multiple
scripts. Using leading numbers when naming scripts allows you to order
them by their position in the workflow.}

\end{figure}

Leaving some decimal places between successive scripts gives you the
ability to easily insert any additional steps you didn't originally
anticipate. And zero-padding means your self-sorting still works once
you move into double-digit numbers.

Other script organization strategies might involve including subfolders
to collect scripts related to main steps. For example, one could have a
subfolder ``04\_classifiers'' to keep alternative classification scripts
in one place, using a more tree-based file structure. Again, each user
or group will develop a system that works for them. The important part
is to have an organizational system.

Tip 3. Consider data types and file sizes when storing intermediates

Images and image collections are common intermediate file types, since
generating satellite composites or creating land use classifications
tends to be computationally intensive. These assets can also be quite
large, depending on the resolution and region size. Recall that our
single-year, subregional Sentinel-2 composite in Sect. 2 was about 23
GB.

Image values can be stored from 8-bit integers to 64-bit double floats
(numbers with decimals). Higher bits allow for more precision, but have
much larger file sizes and are not always necessary. For example, if we
are generating a land use map with five classes, we can convert that to
a signed or unsigned 8-bit integer using toInt8 or toUint8 prior to
exporting to asset, which can accommodate 256 unique values. This
results in a smaller file size. Selectively retaining only bands of
interest is also helpful to reduce size.

For cases requiring decimals and precision, consider whether a 32-bit
float will do the job instead of a 64-bit double---toFloat will convert
an image to a 32-bit float. If you find you need to conserve storage,
you can also scale float values and store as an integer image
(image.multiply(100).toInt16(), for example). This would retain
precision to the second decimal place and reduce file size by a factor
of two. Note that this may require you to unscale the values in
downstream use. Ultimately, the appropriate data type will be specific
to your needs.

And of course, as mentioned above under ``The Importance of Best Coding
Practices,'' be aware of the scale resolution you are working at, and
avoid using unnecessarily high resolution when it's not supported by
either the input imagery or your research goals.

Tip 4. Consider Google Cloud Platform for hosting larger intermediates

If you are working with very large or very many files, you can link
Earth Engine with Cloud Projects on Google Cloud Platform. See the Earth
Engine documentation on ``Setting Up Earth Engine Enabled Cloud
Projects'' for more information.

\hypertarget{conclusion-23}{%
\subsection*{Conclusion}\label{conclusion-23}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to design Earth Engine Apps using both
the Earth Engine User Interface API (JavaScript) and geemap (Python).
You also learned how to deploy Earth Engine Apps on multiple platforms,
such as the JavaScript Code Editor, a local web server, and Heroku. The
skill of designing and deploying interactive Earth Engine Apps is
essential for making your research and data products more accessible to
the scientific community and the general public. Anyone with the link to
your web App can analyze and visualize Earth Engine datasets without
needing an Earth Engine account.

\hypertarget{references-16}{%
\subsection*{References}\label{references-16}}
\addcontentsline{toc}{subsection}{References}

\begin{itemize}
\tightlist
\item
  Earth Engine User Interface API:
  \href{https://www.google.com/url?q=https://developers.google.com/earth-engine/guides/ui\&sa=D\&source=editors\&ust=1671458841355402\&usg=AOvVaw1ZgOA5XvSnn5Uo83ccgHPT}{https://developers.google.com/earth-engine/guides/ui}
\item
  Earth Engine Apps:
  \href{https://www.google.com/url?q=https://developers.google.com/earth-engine/guides/apps\&sa=D\&source=editors\&ust=1671458841355890\&usg=AOvVaw1ZYwRsMtxZ2RPAn-dC-E0n}{https://developers.google.com/earth-engine/guides/apps}
\item
  Voilà:
  \href{https://www.google.com/url?q=https://voila.readthedocs.io\&sa=D\&source=editors\&ust=1671458841356320\&usg=AOvVaw1_h1OkQnXAvfEh717DURMY}{https://voila.readthedocs.io}
\item
  geemap:
  \href{https://www.google.com/url?q=https://geemap.org\&sa=D\&source=editors\&ust=1671458841356762\&usg=AOvVaw3ednZ3gbVzDuptXKRZgKCT}{https://geemap.org}
\item
  ngrok:
  \href{https://www.google.com/url?q=https://ngrok.com\&sa=D\&source=editors\&ust=1671458841357236\&usg=AOvVaw11-kfA1McHDaqztFMUrmtP}{https://ngrok.com}
\item
  Heroku:
  \href{https://www.google.com/url?q=https://heroku.com\&sa=D\&source=editors\&ust=1671458841357713\&usg=AOvVaw15mVX-rHKo54H2oq0Msd2Z}{https://heroku.com}
\item
  Earthengine-apps:
  \href{https://www.google.com/url?q=https://github.com/giswqs/earthengine-apps\&sa=D\&source=editors\&ust=1671458841358120\&usg=AOvVaw2XD4jjXV9SfPmaC-oLbstc}{https://github.com/giswqs/earthengine-apps}
\end{itemize}

Chapter F6.4: Combining R and Earth Engine

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-tip-color!10!white, title=\textcolor{quarto-callout-tip-color}{\faLightbulb}\hspace{0.5em}{Chapter Information}, bottomrule=.15mm, colframe=quarto-callout-tip-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

\hypertarget{author-22}{%
\subsubsection*{Author}\label{author-22}}

Cesar Aybar, David Montero, Antony Barja, Fernando Herrera, Andrea
Gonzales, and Wendy Espinoza

\hypertarget{overview-24}{%
\subsubsection*{Overview}\label{overview-24}}

The purpose of this chapter is to introduce rgee, a non-official API for
Earth Engine. You will explore the main features available in rgee and
how to set up an environment that integrates rgee with third-party R and
Python packages. After this chapter, you will be able to combine R,
Python, and JavaScript in the same workflow.

\hypertarget{learning-outcomes-24}{%
\subsubsection*{Learning Outcomes}\label{learning-outcomes-24}}

\begin{itemize}
\tightlist
\item
  Becoming familiar with rgee, the Earth Engine R API interface.
\item
  Integrating rgee with other R packages.
\item
  Displaying interactive maps.
\item
  Integrating Python and R packages using reticulate.
\item
  Combining Earth Engine JavaScript and Python APIs with R.
\end{itemize}

\hypertarget{assumes-you-know-how-to-24}{%
\subsubsection*{Assumes you know how
to:}\label{assumes-you-know-how-to-24}}

\begin{itemize}
\tightlist
\item
  Install the Python environment (Chap. F6.3).
\item
  Use the require function to load code from existing modules (Chap.
  F6.1).
\item
  Use the basic functions and logic of Python.
\item
  Configure an environment variable and use .Renviron files.
\item
  Create Python virtual environments.
\end{itemize}

\end{tcolorbox}

\hypertarget{introduction-20}{%
\subsection*{Introduction}\label{introduction-20}}

R is a popular programming language established in statistical science
with large support in reproducible research, geospatial analysis, data
visualization, and much more. To get started with R, you will need to
satisfy some extra software requirements. First, install an up-to-date R
version (at least 4.0) in your work environment. The installation
procedure will vary depending on your operating system (i.e., Windows,
Mac, or Linux). Hands-On Programming with R (Garrett Grolemund 2014,
Appendix A) explains step by step how to proceed. We strongly recommend
that Windows users install Rtools to avoid compiling external static
libraries.

If you are new to R, a good starting point is the book Geocomputation
with R (Lovelace et al.~2019) or Spatial Data Science with Application
in R (Pebesma and Bivand 2021). In addition, we recommend using an
integrated development environment (e.g., Rstudio) or a code editor
(e.g., Visual Studio Code) to create a suitable setting to display and
interact with R objects.

The following R packages must be installed (find more information in the
R manual) in order to go through the practicum section.

\hypertarget{use-install.packages-to-install-r-packages-from-the-cran-repository.}{%
\section{Use install.packages to install R packages from the CRAN
repository.}\label{use-install.packages-to-install-r-packages-from-the-cran-repository.}}

install.packages(`reticulate') \# Connect Python with R.\\
install.packages(`rayshader') \# 2D and 3D data visualizations in R.\\
install.packages(`remotes') \# Install R packages from remote
repositories.\\
remotes::install\_github(`r-earthengine/rgeeExtra') \# rgee extended.\\
install.packages(`rgee') \# GEE from within R.\\
install.packages(`sf') \# Simple features in R.\\
install.packages(`stars') \# Spatiotemporal Arrays and Vector Data
Cubes.\\
install.packages(`geojsonio') \# Convert data to `GeoJSON' from various
R classes.\\
install.packages(`raster') \# Reading, writing, manipulating, analyzing
and modeling of spatial data.\\
install.packages(`magick') \# Advanced Graphics and Image-Processing in
R\\
install.packages(`leaflet.extras2') \# Extra Functionality for leaflet\\
install.packages(`cptcity') \# colour gradients from the `cpt-city' web
archive

Earth Engine officially supports client libraries only for the
JavaScript and Python programming languages. While the Earth Engine Code
Editor offers a convenient environment for rapid prototyping in
JavaScript, the lack of a mechanism for integration with local
environments makes the development of complex scripts tricky. On the
other hand, the Python client library offers much versatility, enabling
support for third-party packages. However, not all Earth and
environmental scientists code in Python. Hence, a significant number of
professionals are not members of the Earth Engine community. In the R
ecosystem, rgee (Aybar et al.~2020) tries to fill this gap by wrapping
the Earth Engine Python API via reticulate (Kevin Ushey et al.~2021).
The rgee package extends and supports all the Earth Engine classes,
modules, and functions, working as fast as the other APIs.

\begin{figure}

{\centering \includegraphics{./F6/image48.png}

}

\caption{Fig. F6.4.1 A simplified diagram of rgee functionality}

\end{figure}

Figure F6.4.1 illustrates how rgee bridges the Earth Engine platform
with the R ecosystem. When an Earth Engine request is created in R, rgee
transforms this piece of code into Python. Next, the Earth Engine Python
API converts the Python code into JSON. Finally, the JSON file request
is received by the server through a Web REST API. Users could get the
response using the getInfo method by following the same path in reverse.

\hypertarget{installing-rgee}{%
\subsection{Installing rgee}\label{installing-rgee}}

To run, rgee needs a Python environment with two packages: NumPy and
earthengine-api. Because instructions change frequently, installation is
explained at the following checkpoint:

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64a. The book's repository contains information about
setting up the rgee environment.

\end{tcolorbox}

After installing both the R and Python requirements, you can now
initialize your Earth Engine account from within R. Consider that R, in
contrast to Javascript and Python, supports three distinct Google APIs:
Earth Engine, Google Drive, and Google Cloud Storage (GCS).

The Google Drive and GCS APIs will allow you to transfer your Earth
Engine completed task exports to a local environment automatically. In
these practice sessions, we will use only the Earth Engine and Google
Drive APIs. Users that are interested in GCS can look up and explore the
GCS vignette. To initialize your Earth Engine account alongside Google
Drive, use the following commands.

\hypertarget{initialize-just-earth-engine}{%
\section{Initialize just Earth
Engine}\label{initialize-just-earth-engine}}

ee\_Initialize()

\hypertarget{initialize-earth-engine-and-gdee_initializedrive-true}{%
\section{Initialize Earth Engine and GDee\_Initialize(drive =
TRUE)}\label{initialize-earth-engine-and-gdee_initializedrive-true}}

If the Google account is verified and the permission is granted, you
will be directed to an authentication token. Copy and paste this token
into your R console. Consider that the GCS API requires setting up
credentials manually; look up and explore the rgee vignette for more
information. The verification step is only required once; after that,
rgee saves the credentials in your system.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64b. The book's repository contains information about
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{creating-a-3d-population-density-map-with-rgee-and-rayshader}{%
\subsection{Creating a 3D Population Density Map with rgee and
rayshader}\label{creating-a-3d-population-density-map-with-rgee-and-rayshader}}

First, import the rgee, rayshader, and raster packages.

library(rayshader)\\
library(raster)\\
library(rgee)

Initialize the Earth Engine and Google Drive APIs using ee\_Initialize.
Both credentials must come from the same Google account.

ee\_Initialize(drive = TRUE)

Then, we will access the WorldPop Global Project Population Data
dataset. In rgee, the Earth Engine spatial classes
(ee\(Image, ee\)ImageCollection, and ee\$FeatureCollection) have a
special attribute called Dataset. Users can use it along with
autocompletion to quickly find the desired dataset.

collections \textless- ee\(ImageCollection\)Dataset\\
population\_data \textless-
collections\(CIESIN_GPWv411_GPW_Population_Density population_data_max <- population_data\)max()

If you need more information about the Dataset, use
ee\_utils\_dataset\_display to go to the official documentation in the
Earth Engine Data Catalog.

population\_data \%\textgreater\% ee\_utils\_dataset\_display()

The rgee package provides various built-in functions to retrieve data
from Earth Engine (Aybar et al.~2020). In this example, we use
ee\_as\_raster, which automatically converts an ee\$Image (server
object) into a RasterLayer (local object).

sa\_extent \textless- ee\(Geometry\)Rectangle(\\
coords = c(-100, -50, -20, 12),\\
geodesic = TRUE,\\
proj = ``EPSG:4326'')

population\_data\_ly\_local \textless- ee\_as\_raster(\\
image = population\_data\_max,\\
region = sa\_extent,\\
dsn = ``/home/pc-user01/population.tif'', \# change for your own path.
scale = 5000\\
)

Now, turn a RasterLayer into a matrix suitable for rayshader.

pop\_matrix \textless- raster\_to\_matrix(population\_data\_ly\_local)

Next, modify the matrix population density values, adding:

\begin{itemize}
\tightlist
\item
  Texture, based on five colors (lightcolor, shadowcolor, leftcolor,
  rightcolor, and centercolor; see rayshader::create\_texture
  documentation)
\item
  Color and shadows (rayshader::sphere\_shade)
\end{itemize}

pop\_matrix \%\textgreater\%\\
sphere\_shade(\\
texture = create\_texture(``\#FFFFFF'', ``\#0800F0'', ``\#FFFFFF'',
``\#FFFFFF'', ``\#FFFFFF'')\\
) \%\textgreater\%\\
plot\_3d(\\
pop\_matrix,\\
zoom = 0.55, theta = 0, zscale = 100, soliddepth = -24,\\
solidcolor = ``\#525252'', shadowdepth = -40, shadowcolor = ``black'',\\
shadowwidth = 25, windowsize = c(800, 720)\\
)

Lastly, define a title and subtitle for the plot. Use
rayshader::render\_snapshot to export the final results (Fig. F6.4.2).

text \textless- paste0( ``South Americanpopulation density'',\\
strrep(``n'', 27), ``Source:GPWv411: Population Density (Gridded
Population of the World Version 4.11)'')

render\_snapshot(\\
filename = ``30\_poblacionsudamerica.png'',\\
title\_text = text,\\
title\_size = 20,\\
title\_color = ``black'',\\
title\_font = ``Roboto bold'',\\
clear = TRUE\\
)

\begin{figure}

{\centering \includegraphics{./F6/image2.png}

}

\caption{Fig. F6.4.2 3D population density map of South America}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64c. The book's repository contains information about
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{displaying-maps-interactively}{%
\subsection{Displaying Maps
Interactively}\label{displaying-maps-interactively}}

Similar to the Code Editor, rgee supports the interactive visualization
of spatial Earth Engine objects by Map\$addLayer. First, let's import
the rgee and cptcity packages. The cptcity R package is a wrapper to the
cpt-city color gradients web archive.

library(rgee)\\
library(cptcity)\\
ee\_Initialize()

We'll select an ee\$Image; in this case, the Shuttle Radar Topography
Mission 90 m (SRTM-90) Version 4.

dem \textless- ee\(Image\)Dataset\$CGIAR\_SRTM90\_V4

Then, we'll set the visualization parameters as a list with the
following elements.

\begin{itemize}
\tightlist
\item
  min: value(s) to map to 0
\item
  max: value(s) to map to 1
\item
  palette: a list of CSS-style color strings
\end{itemize}

viz \textless- list(\\
min = 600,\\
max = 6000,\\
palette = cpt(pal = `grass\_elevation', rev = TRUE)\\
)

Then, we'll create a simple display using Map\$addLayer.

m1 \textless- Map\$addLayer(dem, visParams = viz, name = ``SRTM'', shown
= TRUE)

Optionally, you could add a custom legend using Map\$addLayer (Fig.
F6.4.3).

pal \textless- Map\$addLegend(viz)\\
m1 + pal

\begin{figure}

{\centering \includegraphics{./F6/image28.png}

}

\caption{Fig. F6.4.3 Interactive visualization of SRTM-90 Version 4
elevation values}

\end{figure}

The procedure to display ee\(Geometry, ee\)Feature, and
ee\(FeatureCollections objects is similar to the previous example effected on an ee\)Image.
Users just need to change the arguments: eeObject and visParams.

First, Earth Engine geometries (Fig. F6.4.4).

vector \textless- ee\(Geometry\)Point(-77.011,-11.98) \%\textgreater\%\\
ee\(Feature\)buffer(50*1000)\\
Map\(centerObject(vector) Map\)addLayer(vector) \# eeObject is a
ee\(Geometry\)Polygon.

\begin{figure}

{\centering \includegraphics{./F6/image46.png}

}

\caption{Fig. F6.4.4 A polygon buffer surrounding the city of Lima,
Peru}

\end{figure}

Next, Earth Engine feature collections (Fig. F6.4.5).

building \textless- ee\(FeatureCollection\)Dataset\$\\
\texttt{GOOGLE\_Research\_open-buildings\_v1\_polygon}\strut \\
Map\(setCenter(3.389, 6.492, 17) Map\)addLayer(building) \# eeObject is
a ee\$FeatureCollection

\begin{figure}

{\centering \includegraphics{./F6/image23.png}

}

\caption{Fig. F6.4.5 Building footprints in Lagos, Nigeria}

\end{figure}

The rgee functionality also supports the display of
ee\(ImageCollection via Map\)addLayers (note the extra ``s'' at the
end).
Map\(addLayers will use the same visualization parameters for all the images (Fig. F6.4.6). If you need different visualization parameters per image, use a Map\)addLayer
within a for loop.

\hypertarget{define-a-imagecollection}{%
\section{Define a ImageCollection}\label{define-a-imagecollection}}

etp \textless-
ee\(ImageCollection\)Dataset\(MODIS_NTSG_MOD16A2_105 %>%  ee
\)ImageCollection\(select("ET") %>%  ee
\)ImageCollection\$filterDate(`2014-04-01', `2014-06-01')

\hypertarget{set-viz-paramsviz---list}{%
\section{Set viz paramsviz \textless-
list(}\label{set-viz-paramsviz---list}}

min = 0, max = 300,\\
palette = cpt(pal = ``grass\_bcyr'', rev = TRUE)\\
)

\hypertarget{print-map-results-interactively}{%
\section{Print map results
interactively}\label{print-map-results-interactively}}

Map\(setCenter(0, 0, 1) etpmap <- Map\)addLayers(etp, visParams = viz)\\
etpmap

\begin{figure}

{\centering \includegraphics{./F6/image21.png}

}

\caption{Fig. F6.4.6 MOD16A2 total evapotranspiration values
(kg/m\^{}2/8day)}

\end{figure}

Another useful rgee feature is the comparison operator (\textbar), which
creates a slider in the middle of the canvas, permitting quick
comparison of two maps. For instance, load a Landsat 4 image:

landsat \textless-
ee\$Image(`LANDSAT/LT04/C01/T1/LT04\_008067\_19890917')

Calculate the Normalized Difference Snow Index.

ndsi \textless- landsat\$normalizedDifference(c(`B3', `B5'))

Define a constant value and use ee\(Image\)gte to return a binary image
where pixels greater than or equal to that value are set as 1 and the
rest are set as 0. Next, filter 0 values using ee\(Image\)updateMask.

ndsiMasked \textless- ndsi\(updateMask(ndsi\)gte(0.4))

Define the visualization parameters.

vizParams \textless- list(\\
bands \textless- c(`B5', `B4', `B3'), \# vector of three bands (R, G,
B). min = 40,\\
max = 240,\\
gamma = c(0.95, 1.1, 1) \# Gamma correction factor.)

ndsiViz \textless- list(\\
min = 0.5,\\
max = 1,\\
palette = c(`00FFFF', `0000FF')\\
)

Center the map on the Huayhuash mountain range in Peru.

Map\$setCenter(lon = -77.20, lat = -9.85, zoom = 10)

Finally, visualize both maps using the \textbar{} operator (Fig.
F6.4.7).

m2 \textless-
Map\(addLayer(ndsiMasked, ndsiViz, 'NDSI masked') m1 <- Map\)addLayer(landsat,
vizParams, `false color composite')\\
m2 \textbar{} m1

\begin{figure}

{\centering \includegraphics{./F6/image35.png}

}

\caption{Fig. F6.4.7 False-color composite over the Huayhuash mountain
range, Peru}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64d. The book's repository contains information about
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{integrating-rgee-with-other-python-packages}{%
\subsection{Integrating rgee with Other Python
Packages}\label{integrating-rgee-with-other-python-packages}}

As noted in Sect. 1, rgee set up a Python environment with NumPy and
earthengine-api in your system. However, there is no need to limit it to
just two Python packages. In this section, you will learn how to use the
Python package ndvi2gif to perform a Normalized Difference Vegetation
Index (NDVI) multi-seasonal analysis in the Ocoña Valley without leaving
R.

Whenever you want to install a Python package, you must run the
following.

library(rgee)\\
library(reticulate)\\
ee\_Initialize()

The ee\_Initialize function not only authenticates your Earth Engine
account but also helps reticulate to set up a Python environment
compatible with rgee. After running ee\_Initialize, use
reticulate::install\_python to install the desired Python package.

py\_install(``ndvi2gif'')

The previous procedure is needed just once for each Python environment.
Once installed, we simply load the package using reticulate::import.

ngif \textless- import(``ndvi2gif'')

Then, we define our study area using ee\(Geometry\)Rectangle (Fig.
F6.4.8), and use the leaflet layers control to switch between basemaps.

colca \textless- c(-73.15315, -16.46289, -73.07465, -16.37857)\\
roi \textless- ee\(Geometry\)Rectangle(colca)\\
Map\(centerObject(roi) Map\)addLayer(roi)

\begin{figure}

{\centering \includegraphics{./F6/image59.png}

}

\caption{Fig. F6.4.8 A rectangle drawn over the Ocoña Valley, Peru}

\end{figure}

In ndvi2gif, there is just one class: NdviSeasonality. It has the
following four public methods.

\begin{itemize}
\tightlist
\item
  get\_export: Exports NDVI year composites in .GeoTIFF format to your
  local folder.
\item
  get\_export\_single: Exports single composite as .GeoTIFF to your
  local folder.
\item
  get\_year\_composite: Returns the NDVI composites for each year.
\item
  get\_gif: Exports NDVI year composites as a .gif to your local folder.
\end{itemize}

To run, the NdviSeasonality constructor needs to define the following
arguments.

\begin{itemize}
\tightlist
\item
  roi: the region of interest
\item
  start\_year: the initial year to start to create yearly composites
\item
  end\_year: the end year to look for
\item
  sat: the satellite sensor
\item
  key: the aggregation rule that will be used to generate the yearly
  composite
\end{itemize}

For each year, the get\_year\_composite method generates an NDVI
ee\$Image with four bands, one band per season. Color combination
between images and bands will allow you to interpret the vegetation
phenology over the seasons and years. In ndvi2gif, the seasons are
defined as follows.

\begin{itemize}
\tightlist
\item
  winter = c(`-01-01', `-03-31')
\item
  spring = c(`-04-01', `-06-30')
\item
  summer = c(`-07-01', `-09-30')
\item
  autumn = c(`-10-01', `-12-31')
\end{itemize}

myclass \textless- ngif\$NdviSeasonality(\\
roi = roi,\\
start\_year = 2016L,\\
end\_year = 2020L,\\
sat = `Sentinel', \# `Sentinel', `Landsat', `MODIS', `sar' key = `max'
\# `max', `median', `perc\_90'\\
)

\hypertarget{estimate-the-median-of-the-yearly-composites-from-2016-to-2020.}{%
\section{Estimate the median of the yearly composites from 2016 to
2020.}\label{estimate-the-median-of-the-yearly-composites-from-2016-to-2020.}}

median \textless- myclass\(get_year_composite()\)median()

\hypertarget{estimate-the-median-of-the-winter-season-composites-from-2016-to-2020.}{%
\section{Estimate the median of the winter season composites from 2016
to
2020.}\label{estimate-the-median-of-the-winter-season-composites-from-2016-to-2020.}}

wintermax \textless-
myclass\(get_year_composite()\)select(`winter')\$max()

We can display maps interactively using the Map\$addLayer (Fig. F6.4.9),
and use the leaflet layers control to switch between basemaps.

Map\(addLayer(wintermax, list(min = 0.1, max = 0.8), 'winterMax') | Map\)addLayer(median,
list(min = 0.1, max = 0.8), `median')

\begin{figure}

{\centering \includegraphics{./F6/image24.png}

}

\caption{Fig. F6.4.9 Comparison between the maximum historic winter NDVI
and the mean historic NDVI. Colors represent the season when the maximum
value occurred.}

\end{figure}

And we can export the results to a GIF format.

myclass\$get\_gif()

To get more information about the ndvi2gif package, visit its
\href{https://www.google.com/url?q=https://github.com/Digdgeo/Ndvi2Gif\&sa=D\&source=editors\&ust=1671458841429620\&usg=AOvVaw0cUiJ9wlrQB8a2-Whxs0zA}{GitHub}
repository.

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64e. The book's repository contains information about
what your code should look like at this point.

\end{tcolorbox}

\hypertarget{converting-javascript-modules-to-r}{%
\subsection{Converting JavaScript Modules to
R}\label{converting-javascript-modules-to-r}}

In recent years, the Earth Engine community has developed a lot of
valuable third-party modules. Some incredible ones are geeSharp (Zuspan
2020), ee-palettes (Donchyts et al.~2020), spectral (Montero 2021), and
LandsatLST (Ermida et al.~2020). While some of these modules have been
implemented in Python and JavaScript (e.g., geeSharp and spectral), most
are available only for JavaScript. This is a critical drawback, because
it divides the Earth Engine community by programming languages. For
example, if an R user wants to use tagee (Safanelli et al.~2020), the
user will have to first translate the entire module to R.

In order to close this breach, the ee\_extra Python package has been
developed to unify the Earth Engine community. The philosophy behind
ee\_extra is that all of its extended functions, classes, and methods
must be functional for the JavaScript, Julia, R, and Python client
libraries. Currently, ee\_extra is the base of the rgeeExtra (Aybar et
al.~2021) and eemont (Montero 2021) packages.

To demonstrate the potential of ee\_extra, let's study an example from
the Landsat Land Surface Temperature (LST) JavaScript module. The
Landsat LST module computes the land surface temperature for Landsat
products (Ermida et al.~2020). First we will run it in the Earth Engine
Code Editor; then we will replicate those results in R.

First, JavaScript. In a new script in the Code Editor, we must require
the Landsat LST module.

var LandsatLST = require(
`users/sofiaermida/landsat\_smw\_lst:modules/Landsat\_LST.js');

The Landsat LST module contains a function named collection. This
function receives the following parameters.

\begin{itemize}
\tightlist
\item
  The Landsat archive ID
\item
  The starting date of the Landsat collection
\item
  The ending date of the Landsat collection
\item
  The region of interest as geometry
\item
  A Boolean parameter specifying if we want to use the NDVI for
  computing a dynamic emissivity instead of using the emissivity from
  ASTER
\end{itemize}

In the following code block, we are going to define all required
parameters.

var geometry = ee.Geometry.Rectangle({[}-8.91, 40.0, -8.3, 40.4{]});\\
var satellite = `L8';\\
var date\_start = `2018-05-15';\\
var date\_end = `2018-05-31';\\
var use\_ndvi = true;

Now, with all our parameters defined, we can compute the land surface
temperature by using the collection method from Landsat LST.

var LandsatColl = LandsatLST.collection(satellite, date\_start,\\
date\_end, geometry, use\_ndvi);

The result is stored as an ImageCollection in the LandsatColl variable.
Now select the first element of the collection as an example by using
the first method.

var exImage = LandsatColl.first();

This example image is now stored in a variable named `exImage'. Let's
display the LST result on the Map. For visualization purposes, we'll
define a color palette.

var cmap = {[}`blue', `cyan', `green', `yellow', `red'{]};

Then, we'll center the map in the region of interest.

Map.centerObject(geometry);

Finally, let's display the LST with the cmap color palette by using the
Map.addLayer method (Fig. F6.4.10). This method receives the image to
visualize, the visualization parameters, the color palette, and the name
of the layer to show in the layer control. The visualization parameters
will be:

\begin{itemize}
\tightlist
\item
  min: 290 (a minimum LST value of 290 K)
\item
  max: 320 (a maximum LST value of 320 K)
\item
  palette: cmap (the color palette that was created some steps before)
\end{itemize}

The name of the layer in the Map layer set will be LST.

Map.addLayer(exImage.select(`LST'), \{\\
min: 290,\\
max: 320,\\
palette: cmap\\
\}, `LST')

\begin{figure}

{\centering \includegraphics{./F6/image45.png}

}

\caption{Fig. F6.4.10 A map illustrating LST, obtained by following the
JavaScript example}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64f. The book's repository contains a script that shows
what your code should look like at this point.

\end{tcolorbox}

Now, let's use R to implement the same logic. As in the previous
sections, import the R packages: rgee and rgeeExtra. Then, initialize
your Earth Engine session.

library(rgee)\\
library(rgeeExtra)\\
library(reticulate)

ee\_Initialize()

Install rgeeExtra Python dependencies.

py\_install(packages = c(``regex'', ``ee\_extra'', ``jsbeautifier''))

Using the function rgeeExtra::module loads the JavaScript module.

LandsatLST \textless-
module(``users/sofiaermida/landsat\_smw\_lst:modules/Landsat\_LST.js'')

The rest of the code is exactly the same as in JavaScript.

geometry \textless- ee\(Geometry\)Rectangle(c(-8.91, 40.0, -8.3,
40.4))\\
satellite \textless- 'L8'date\_start \textless- '2018-05-15'date\_end
\textless- '2018-05-31'use\_ndvi \textless- TRUE

LandsatColl \textless-
LandsatLST\(collection(satellite, date_start, date_end, geometry, use_ndvi) exImage <- LandsatColl\)first()\\
cmap \textless- c(`blue', `cyan', `green', `yellow', `red')

lmod \textless- list(min = 290, max = 320, palette = cmap)\\
Map\(centerObject(geometry) Map\)addLayer(exImage\$select(`LST'), lmod,
`LST')

\begin{figure}

{\centering \includegraphics{./F6/image7.png}

}

\caption{Fig. F6.4.11 A map illustrating LST, obtained by following the
R example}

\end{figure}

\begin{tcolorbox}[enhanced jigsaw, opacityback=0, breakable, toptitle=1mm, left=2mm, opacitybacktitle=0.6, toprule=.15mm, titlerule=0mm, colbacktitle=quarto-callout-note-color!10!white, title=\textcolor{quarto-callout-note-color}{\faInfo}\hspace{0.5em}{Note}, bottomrule=.15mm, colframe=quarto-callout-note-color-frame, rightrule=.15mm, coltitle=black, leftrule=.75mm, arc=.35mm, colback=white, bottomtitle=1mm]

Code Checkpoint F64g. The book's repository contains information about
what your code should look like at this point.

\end{tcolorbox}

Question 1. When and why might users prefer to use R instead of Python
to connect to Earth Engine?

Question 2. What are the advantages and disadvantages of using rgee
instead of the Earth Engine JavaScript Code Editor?

\hypertarget{conclusion-24}{%
\subsection*{Conclusion}\label{conclusion-24}}
\addcontentsline{toc}{subsection}{Conclusion}

In this chapter, you learned how to use Earth Engine and R in the same
workflow. Since rgee uses reticulate, rgee also permits integration with
third-party Earth Engine Python packages. You also learned how to use
Map\$addLayer, which works similarly to the Earth Engine User Interface
API (Code Editor). Finally, we also introduced rgeeExtra, a new R
package that extends the Earth Engine API and supports JavaScript module
execution.

\hypertarget{references-17}{%
\subsection*{References}\label{references-17}}
\addcontentsline{toc}{subsection}{References}

Aybar C, Wu Q, Bautista L, et al (2020) rgee: An R package for
interacting with Google Earth Engine. J Open Source Softw 5:2272.
https://doi.org/10.21105/joss.02272

Ermida SL, Soares P, Mantas V, et al (2020) Google Earth Engine
open-source code for land surface temperature estimation from the
Landsat series. Remote Sens 12:1471. https://doi.org/10.3390/RS12091471

Grolemund G (2014) Hands-On Programming with R - Write Your Own
Functions and Simulations. O'Reilly Media, Inc.

Lovelace R, Nowosad J, Muenchow J (2019) Geocomputation with R. Chapman
and Hall/CRC

Montero D (2021) eemont: A Python package that extends Google Earth
Engine. J Open Source Softw 6:3168. https://doi.org/10.21105/joss.03168

Pebesma E, Bivand R (2019) Spatial Data Science.
https://r-spatial.org/book/

\part{Case Studies}

\hypertarget{war-at-night}{%
\chapter*{War at Night}\label{war-at-night}}
\addcontentsline{toc}{chapter}{War at Night}

\markboth{War at Night}{War at Night}

\hypertarget{data}{%
\section*{Data}\label{data}}
\addcontentsline{toc}{section}{Data}

\markright{Data}

Satellite images of Syria taken at night capture a subtle trace left by
human civilization: lights. Apartment buildings, street lights,
highways, powerplants-- all are illuminated at night and can be seen
from space. Researchers often use these nighttime lights signatures to
track development; as cities grow, villages recieve power, and
infrastructure is built, areas emit more light. But this works both
ways. As cities are demolished, villages burned, and highways cutoff,
they stop emitting lights.

The timelapse below uses imagery from the Defense Meteorological
Satellite Program (DMSP), a joint program run by the U.S. Department of
Defense and the National Oceanographic and Atmospheric Agency. One image
is taken per year between 2005 and 2013:

\hypertarget{ukraine}{%
\section*{Ukraine}\label{ukraine}}
\addcontentsline{toc}{section}{Ukraine}

\markright{Ukraine}

\hypertarget{pre-processing}{%
\subsection*{Pre-Processing}\label{pre-processing}}
\addcontentsline{toc}{subsection}{Pre-Processing}

\hypertarget{analysis}{%
\subsection*{Analysis}\label{analysis}}
\addcontentsline{toc}{subsection}{Analysis}

\hypertarget{iraq}{%
\section*{Iraq}\label{iraq}}
\addcontentsline{toc}{section}{Iraq}

\markright{Iraq}

A link to the GEE code for this section can be found
\href{https://code.earthengine.google.com/2cf77d8cb9afd76b73100637fbffdf5d}{here}.

\hypertarget{pre-processing-1}{%
\subsection*{Pre-Processing}\label{pre-processing-1}}
\addcontentsline{toc}{subsection}{Pre-Processing}

First, let's start by importing a few useful packages written by
\href{https://twitter.com/gena_d}{Gennadii Donchyts}. We'll use
\texttt{utils} and \texttt{text} to annotate the date of each image on
the timelapse. We'll also define an Area of Interest (AOI), which is
just a rectangle. You can do this manually by clicking the drawing tools
in the top left. I've drawn an AOI over the area covering Mosul, Irbil,
and Kirkuk in Northern Iraq.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ utils }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{"users/gena/packages:utils"}\NormalTok{)}\OperatorTok{;}
\KeywordTok{var}\NormalTok{ text }\OperatorTok{=} \PreprocessorTok{require}\NormalTok{(}\StringTok{"users/gena/packages:text"}\NormalTok{)}\OperatorTok{;}

\CommentTok{// define the Area of Interest (AOI)}
\KeywordTok{var}\NormalTok{ AOI }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{(}
\NormalTok{        [[[}\FloatTok{42.555362833405326}\OperatorTok{,} \FloatTok{36.62010778397765}\NormalTok{]}\OperatorTok{,}
\NormalTok{          [}\FloatTok{42.555362833405326}\OperatorTok{,} \FloatTok{35.18296243288332}\NormalTok{]}\OperatorTok{,}
\NormalTok{          [}\FloatTok{44.681217325592826}\OperatorTok{,} \FloatTok{35.18296243288332}\NormalTok{]}\OperatorTok{,}
\NormalTok{          [}\FloatTok{44.681217325592826}\OperatorTok{,} \FloatTok{36.62010778397765}\NormalTok{]]])}

\CommentTok{// start and end dates for our gif }
\KeywordTok{var}\NormalTok{ startDate }\OperatorTok{=} \StringTok{\textquotesingle{}2013{-}01{-}01\textquotesingle{}}\OperatorTok{;}
\KeywordTok{var}\NormalTok{ endDate }\OperatorTok{=} \StringTok{\textquotesingle{}2018{-}01{-}01\textquotesingle{}}\OperatorTok{;}

\CommentTok{// a filename for when we export the gif}
\KeywordTok{var}\NormalTok{ export\_name}\OperatorTok{=}\StringTok{\textquotesingle{}qayyarah\_viirs\textquotesingle{}}
          
\CommentTok{// A palette to visualize the VIIRS imagery. This one is similar to Matplotlib\textquotesingle{}s "Magma" palette. }
\KeywordTok{var}\NormalTok{ viirs\_palette }\OperatorTok{=}\NormalTok{ [}
  \StringTok{"\#000004"}\OperatorTok{,}
  \StringTok{"\#320a5a"}\OperatorTok{,}
  \StringTok{"\#781b6c"}\OperatorTok{,}
  \StringTok{"\#bb3654"}\OperatorTok{,}
  \StringTok{"\#ec6824"}\OperatorTok{,}
  \StringTok{"\#fbb41a"}\OperatorTok{,}
  \StringTok{"\#fcffa4"}\OperatorTok{,}
\NormalTok{]}\OperatorTok{;}

\CommentTok{// Visualisation parameters for the VIIRS imagery, defining a minimum and maximum value, and referencing the palette we just created}
\KeywordTok{var}\NormalTok{ VIIRSvis }\OperatorTok{=}\NormalTok{ \{ }\DataTypeTok{min}\OperatorTok{:} \OperatorTok{{-}}\FloatTok{0.1}\OperatorTok{,} \DataTypeTok{max}\OperatorTok{:} \FloatTok{1.6}\OperatorTok{,} \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ viirs\_palette \}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, we'll load the VIIRS nighttime lights imagery. We want to select
the \texttt{avg\_rad} band of the image collection, and filter blank
images. Sometimes, we get blank images over an area in VIIRS if our AOI
is on the edge of the satellite's imaging swath. We can filter these
images, similarly to how we filter for cloudy images in Sentinel-2:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ VIIRS}\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{"NOAA/VIIRS/DNB/MONTHLY\_V1/VCMCFG"}\NormalTok{) }
                    \OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{\textquotesingle{}avg\_rad\textquotesingle{}}\NormalTok{)}
                    \CommentTok{// Calculate the sum of the \textquotesingle{}avg\_rad\textquotesingle{} band within the AOI}
                    \OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(image) \{ }
                    \KeywordTok{var}\NormalTok{ blank}\OperatorTok{=}\NormalTok{image}\OperatorTok{.}\FunctionTok{reduceRegions}\NormalTok{(\{ }\CommentTok{// reduceRegions is a function that allows us to reduce the values of a band within a}
                                    \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ AOI}\OperatorTok{,} \CommentTok{// geometry. In this case, we\textquotesingle{}re reducing the values of the \textquotesingle{}avg\_rad\textquotesingle{} band within the AOI}
                                    \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{sum}\NormalTok{()}\OperatorTok{,} \CommentTok{// We\textquotesingle{}re using the sum reducer, which will sum the values of the \textquotesingle{}avg\_rad\textquotesingle{} band}
                                    \DataTypeTok{scale}\OperatorTok{:} \DecValTok{10}\NormalTok{\}) }\CommentTok{// We\textquotesingle{}re reducing the values of the \textquotesingle{}avg\_rad\textquotesingle{} band at a scale of 10m}
                                \OperatorTok{.}\FunctionTok{first}\NormalTok{() }\CommentTok{// We only want the first element of the collection, which is the sum of the \textquotesingle{}avg\_rad\textquotesingle{} band within the AOI}
                                \OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{\textquotesingle{}sum\textquotesingle{}}\NormalTok{) }\CommentTok{// We want the value of the \textquotesingle{}sum\textquotesingle{} property, which is the sum of the \textquotesingle{}avg\_rad\textquotesingle{} band within the AOI}
                    \CommentTok{// For each image, define a property \textquotesingle{}blank\textquotesingle{} that stores the sum of the \textquotesingle{}avg\_rad\textquotesingle{} band within the AOI. }
                    \CommentTok{// We\textquotesingle{}re also going to take a base 10 log of the image{-}{-} this will help us visualize the data by dampening extreme values             }
                    \ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{\textquotesingle{}blank\textquotesingle{}}\OperatorTok{,}\NormalTok{ blank)}\OperatorTok{.}\FunctionTok{log10}\NormalTok{()}\OperatorTok{.}\FunctionTok{unmask}\NormalTok{(}\DecValTok{0}\NormalTok{)}
\NormalTok{                    \})}
                    \CommentTok{// Now, we can filter images which are fully or partially blank over our AOI}
                    \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\StringTok{\textquotesingle{}blank\textquotesingle{}}\OperatorTok{,} \DecValTok{10}\NormalTok{))}
                    \CommentTok{// Finally, we filter the collection to the specified date range}
                    \OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(startDate}\OperatorTok{,}\NormalTok{ endDate)}
                    
\end{Highlighting}
\end{Shaded}

Let's have a look at the first image in the collection to make sure
everything's looking right. We'll set the basemap to satellite and
center our AOI:

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{\textquotesingle{}HYBRID\textquotesingle{}}\NormalTok{)}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{centerObject}\NormalTok{(AOI)}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(VIIRS}\OperatorTok{.}\FunctionTok{first}\NormalTok{()}\OperatorTok{,}\NormalTok{VIIRSvis}\OperatorTok{,}\StringTok{\textquotesingle{}Nighttime Lights\textquotesingle{}}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\includegraphics{././images/iraq_check.png}

If we decrease the opacity of the VIIRS layer, we can see the cities of
Mosul, Erbil, and Kirkuk shining brightly at night. We can also see a
string of bright lights between Kirkuk and Erbil-- these are methane
flares from oil wells.

\hypertarget{analysis-1}{%
\subsection*{Analysis}\label{analysis-1}}
\addcontentsline{toc}{subsection}{Analysis}

Having pre-processed the VIIRS imagery, we can now define a function
\texttt{gif} that will take:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  An image collection (\texttt{col}, in this case the nighttime lights
  imagery \texttt{VIIRS})
\item
  Visualization parameters (\texttt{col\_vis}, in this case
  \texttt{VIIRSvis})
\item
  An Area of Interest \texttt{AOI}
\end{enumerate}

The function will then return a timelapse.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ gif }\OperatorTok{=} \KeywordTok{function}\NormalTok{ (col}\OperatorTok{,}\NormalTok{ col\_vis}\OperatorTok{,}\NormalTok{ AOI) \{}

    \CommentTok{// Define the date annotations to be printed in the top left of the gif in white}
  \KeywordTok{var}\NormalTok{ annotations }\OperatorTok{=}\NormalTok{ [}
\NormalTok{    \{}
      \DataTypeTok{textColor}\OperatorTok{:} \StringTok{"white"}\OperatorTok{,}
      \DataTypeTok{position}\OperatorTok{:} \StringTok{"left"}\OperatorTok{,}
      \DataTypeTok{offset}\OperatorTok{:} \StringTok{"1\%"}\OperatorTok{,}
      \DataTypeTok{margin}\OperatorTok{:} \StringTok{"1\%"}\OperatorTok{,}
      \DataTypeTok{property}\OperatorTok{:} \StringTok{"label"}\OperatorTok{,}
      \CommentTok{// Dynamically size the annotations according to the size of the AOI}
      \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ AOI}\OperatorTok{.}\FunctionTok{area}\NormalTok{(}\DecValTok{100}\NormalTok{)}\OperatorTok{.}\FunctionTok{sqrt}\NormalTok{()}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(}\DecValTok{200}\NormalTok{)}\OperatorTok{,}
\NormalTok{    \}}\OperatorTok{,}
\NormalTok{  ]}\OperatorTok{;}

    \CommentTok{// Next, we want to map over the image collection,}
    \KeywordTok{var}\NormalTok{ rgbVis }\OperatorTok{=}\NormalTok{ col}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{ (image) \{}
        \CommentTok{// Get the date of the image and format it}
        \KeywordTok{var}\NormalTok{ start }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(image}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{"system:time\_start"}\NormalTok{))}\OperatorTok{;}
        \KeywordTok{var}\NormalTok{ label }\OperatorTok{=}\NormalTok{ start}\OperatorTok{.}\FunctionTok{format}\NormalTok{(}\StringTok{"YYYY{-}MM{-}dd"}\NormalTok{)}\OperatorTok{;}
        \CommentTok{// And visualize the image using the visualization parameters defined earlier.}
        \CommentTok{// We also want to set a property called "label" that stores the formatted date }
        \ControlFlowTok{return}\NormalTok{ image}\OperatorTok{.}\FunctionTok{visualize}\NormalTok{(col\_vis)}\OperatorTok{.}\FunctionTok{set}\NormalTok{(\{ }\DataTypeTok{label}\OperatorTok{:}\NormalTok{ label \})}\OperatorTok{;}
\NormalTok{  \})}\OperatorTok{;}

    \CommentTok{// Now we use the label proprty and the annotateImage function from @gena\_d to annotate each image with the date. }
\NormalTok{  rgbVis }\OperatorTok{=}\NormalTok{ rgbVis}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{ (image) \{}
    \ControlFlowTok{return}\NormalTok{ text}\OperatorTok{.}\FunctionTok{annotateImage}\NormalTok{(image}\OperatorTok{,}\NormalTok{ \{\}}\OperatorTok{,}\NormalTok{ AOI}\OperatorTok{,}\NormalTok{ annotations)}\OperatorTok{;}
\NormalTok{  \})}\OperatorTok{;}

  \CommentTok{// Define GIF visualization parameters.}
  \KeywordTok{var}\NormalTok{ gifParams }\OperatorTok{=}\NormalTok{ \{}
    \DataTypeTok{maxPixels}\OperatorTok{:} \DecValTok{27017280}\OperatorTok{,}
    \DataTypeTok{region}\OperatorTok{:}\NormalTok{ AOI}\OperatorTok{,}
    \DataTypeTok{crs}\OperatorTok{:} \StringTok{"EPSG:3857"}\OperatorTok{,}
    \DataTypeTok{dimensions}\OperatorTok{:} \DecValTok{640}\OperatorTok{,}
    \DataTypeTok{framesPerSecond}\OperatorTok{:} \DecValTok{5}\OperatorTok{,}
\NormalTok{  \}}\OperatorTok{;}

  \CommentTok{// Export the gif to Google Drive}
\NormalTok{  Export}\OperatorTok{.}\AttributeTok{video}\OperatorTok{.}\FunctionTok{toDrive}\NormalTok{(\{}
    \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ rgbVis}\OperatorTok{,} \CommentTok{// the image collection}
    \DataTypeTok{description}\OperatorTok{:}\NormalTok{ export\_name}\OperatorTok{,} \CommentTok{// the name of the file}
    \DataTypeTok{dimensions}\OperatorTok{:} \DecValTok{1080}\OperatorTok{,} \CommentTok{// the dimensions of the gif}
    \DataTypeTok{framesPerSecond}\OperatorTok{:} \DecValTok{5}\OperatorTok{,} \CommentTok{// the number of frames per second}
    \DataTypeTok{region}\OperatorTok{:}\NormalTok{ AOI}\OperatorTok{,} \CommentTok{// the area of interest}
\NormalTok{  \})}\OperatorTok{;}
  \CommentTok{// Print the GIF URL to the console.}
  \FunctionTok{print}\NormalTok{(rgbVis}\OperatorTok{.}\FunctionTok{getVideoThumbURL}\NormalTok{(gifParams))}\OperatorTok{;}

  \CommentTok{// Render the GIF animation in the console.}
  \FunctionTok{print}\NormalTok{(ui}\OperatorTok{.}\FunctionTok{Thumbnail}\NormalTok{(rgbVis}\OperatorTok{,}\NormalTok{ gifParams))}\OperatorTok{;}
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Ok that was a pretty big chunk of code. But the good news is that we
basically never have to touch it again, since we can just feed it
different inputs. For example, if I want to generate a gif of nighttime
lights over a different area, it's as simple as dragging the AOI. If I
want to look at a different time period, I can just edit the
\texttt{startDate} and \texttt{endDate} variables. And if I want to
visualize an entirely different type of satellite imagery-- Sentinel-1,
Sentinel-2, or anything else, all I have to do is change the image
collection (\texttt{col}) and visualization parameters
(\texttt{col\_vis}) variables. Now, let's look at some timelapses.

\hypertarget{the-fall-of-mosul}{%
\subsubsection*{The Fall of Mosul}\label{the-fall-of-mosul}}
\addcontentsline{toc}{subsubsection}{The Fall of Mosul}

The function returns a timelapse of nighttime lights over Northern Iraq:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{gif}\NormalTok{(VIIRS}\OperatorTok{,}\NormalTok{ VIIRSvis}\OperatorTok{,}\NormalTok{ AOI)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{././images/Figure_1.gif}

}

\caption{I've done a bit of post-processing to this gif, adding more
annotations and blending between frames to make it a bit smoother. I
typically use \href{https://ffmpeg.org/}{ffmpeg} and
\href{https://ezgif.com/}{ezgif} for the finishing touches.}

\end{figure}

This timelapse gives a play-by-play of one of the most important
campaigns in the war against the Islamic State. In the first few frames,
Mosul is under the control of the Kurdistan Regional Government (KRG).
In the summer of 2014, ISIS captures the city, and power is cut off.
Mosul and many villages along the Tigris river are plunged into
darkness. In 2015, the front line in the campaign to retake the city
emerges around Mosul, advancing in 2016 and 2017. Mosul is eventually
retaken by the KRG in 2017, after which it brightens once again as
electricity is restored.

\hypertarget{the-qayyarah-fires}{%
\subsubsection*{The Qayyarah Fires}\label{the-qayyarah-fires}}
\addcontentsline{toc}{subsubsection}{The Qayyarah Fires}

Farther south, there is an interesting detail. Above the ``h'' in
``Qayyarah'', a bright set of lights emerges just before Mosul is
recaptured, around December 2016. Fleeing Islamic State fighters
\href{https://time.com/iraq-fires/}{set fire to the Qayyarah oilfields},
which burned for months.

Using the VIIRS data we've already loaded, we can further analyze the
effect of the conflict using a chart. First, let's define two rectangles
(again, you can draw these) over Mosul and Qayyarah:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ mosul }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Feature}\NormalTok{(}
\NormalTok{        ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{(}
\NormalTok{            [[[}\FloatTok{43.054977780266675}\OperatorTok{,} \FloatTok{36.438274276521234}\NormalTok{]}\OperatorTok{,}
\NormalTok{              [}\FloatTok{43.054977780266675}\OperatorTok{,} \FloatTok{36.290642221212416}\NormalTok{]}\OperatorTok{,}
\NormalTok{              [}\FloatTok{43.24792516796199}\OperatorTok{,} \FloatTok{36.290642221212416}\NormalTok{]}\OperatorTok{,}
\NormalTok{              [}\FloatTok{43.24792516796199}\OperatorTok{,} \FloatTok{36.438274276521234}\NormalTok{]]]}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{false}\NormalTok{)}\OperatorTok{,}
\NormalTok{        \{}
          \StringTok{"label"}\OperatorTok{:} \StringTok{"Mosul"}\OperatorTok{,}
          \StringTok{"system:index"}\OperatorTok{:} \StringTok{"0"}
\NormalTok{        \})}\OperatorTok{,}

\NormalTok{    qayyarah }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Feature}\NormalTok{(}
\NormalTok{        ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{(}
\NormalTok{            [[[}\FloatTok{43.08240275545117}\OperatorTok{,} \FloatTok{35.8925587996721}\NormalTok{]}\OperatorTok{,}
\NormalTok{              [}\FloatTok{43.08240275545117}\OperatorTok{,} \FloatTok{35.77899970860588}\NormalTok{]}\OperatorTok{,}
\NormalTok{              [}\FloatTok{43.26642375154492}\OperatorTok{,} \FloatTok{35.77899970860588}\NormalTok{]}\OperatorTok{,}
\NormalTok{              [}\FloatTok{43.26642375154492}\OperatorTok{,} \FloatTok{35.8925587996721}\NormalTok{]]]}\OperatorTok{,} \KeywordTok{null}\OperatorTok{,} \KeywordTok{false}\NormalTok{)}\OperatorTok{,}
\NormalTok{        \{}
          \StringTok{"label"}\OperatorTok{:} \StringTok{"Qayyarah"}\OperatorTok{,}
          \StringTok{"system:index"}\OperatorTok{:} \StringTok{"0"}
\NormalTok{        \})}

\CommentTok{// Let\textquotesingle{}s put these together in a list }
\KeywordTok{var}\NormalTok{ regions}\OperatorTok{=}\NormalTok{[qayyarah}\OperatorTok{,}\NormalTok{ mosul]}
\end{Highlighting}
\end{Shaded}

Once we've got the rectangles, we can make a chart that will take the
mean value of the VIIRS images in each rectangle over time:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ chart }\OperatorTok{=}
\NormalTok{    ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{image}
        \OperatorTok{.}\FunctionTok{seriesByRegion}\NormalTok{(\{}
          \DataTypeTok{imageCollection}\OperatorTok{:}\NormalTok{ VIIRS}\OperatorTok{,}
          \DataTypeTok{regions}\OperatorTok{:}\NormalTok{ regions}\OperatorTok{,}
          \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,}
          \DataTypeTok{seriesProperty}\OperatorTok{:}\StringTok{\textquotesingle{}label\textquotesingle{}}
\NormalTok{        \})}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{}
          \DataTypeTok{title}\OperatorTok{:} \StringTok{\textquotesingle{}Nighttime Lights\textquotesingle{}}
\NormalTok{        \})}\OperatorTok{;}
        
\FunctionTok{print}\NormalTok{(chart)}
\end{Highlighting}
\end{Shaded}

\includegraphics{././images/qayyarah_chart.png}

We can clearly see Mosul (the red line) darkening in 2014 as the city is
taken by ISIS. During this period the Qayyarah oilfileds are, as we
might expect, quite dark. All of a sudden in 2016 Qayyarah becomes
brighter at night than the city of Mosul ever was, as the oilfields are
set on fire. Then, almost exactly when the blaze in Qayyarah is
extinguished and the area darkens (i.e.~when the blue line falls back to
near zero), Mosul brightens once again (i.e.~the red line rises) as the
city is liberated.

\hypertarget{the-battle-for-aleppo}{%
\subsection*{The Battle for Aleppo}\label{the-battle-for-aleppo}}
\addcontentsline{toc}{subsection}{The Battle for Aleppo}

The images below were taken between 2012 and 2014. Vast swaths of the
city darken as neighbourhoods are razed by fighting.

Though this is a trend that can be observed across the country, nowhere
is the decline in nightlights more visible than in Aleppo. Below is a
comparison of longitudinal trends in nighlights signatures between
several cities:

The most salient trend is Aleppo plummeting over the course of 2012, and
becoming steadily darker over the course of the next four years. Raqqa
drops in 2012 as well, but remains in flux until 2017, when the battle
to reclaim the city pluges it into near total darkness. Damascus also
experiences a dip in 2012, but stabilizes relatively quickly. The
Turkish city of Gaziantep-- less than 100km from Aleppo and roughly
1/5th the size-- stands in stark contrast to the Syrian cities, becoming
progressively brighter over the entire period.

Another interesting pattern here is the difference in seasonal trends in
nightlights. Under normal circumstances in this part of the world,
cities become brighter at night during the summer months. Restaurants,
bars, and markets stay open later and conduct business outdoors.
Gaziantep, which still attracts scores of tourists every year, displays
pronounced seasonality. Damascus, the most stable of the three Syrian
cities, also maintains a seasonal trend throughout the war. In contrast,
both Raqqa and Aleppo maintain extremely low and roughly constant levels
of nightlights year-round during the periods following intense fighting.

Reliable economic data for Syria haven't been available for nearly a
decade, and assessing the country's recovery is consequently difficult.
But subtle indications of economic growth are visible above: all three
Syrian cities have been on a steady upward trend since 2017, and
beginning to display seasonal variation once again.

\hypertarget{fighting-for-oil}{%
\subsection*{Fighting for Oil}\label{fighting-for-oil}}
\addcontentsline{toc}{subsection}{Fighting for Oil}

Throughout the war, sudden massive spikes in nightlights signatures can
be observed throughout the country. In the center of the map just west
of Palmyra, some particularly large spikes occur in 2017:

These flashes of light show gas wells being set on fire, a common form
of sabotage carried out by retreating Islamic State fighters. Modified
Sentinel-2 imagery of the Hayyan gas field (indicated by the green box
above) shows this in greater detail. Substituing the Red band in an RGB
image with Near Infrared (NIR) highlights thermal signatures, showing
fires burning brightly even during the day.

The large complex on the right is the Hayyan Gas Plant, which produced
nearly 1/3 of Syria's electricity. The plant and its associated wells
changed hands several times throughout the war, but were under Islamic
State control until February 2017. In the video below, Islamic State
fighters can be seen rigging the plant with explosives and destroying it
on January 8th:

In February, three Russian oil and gas companies (Zarubij Naft, Lukoil
and Gazprom Neft) were given restoration, exploration, and production
rights to the hydrocarbon deposits West of Palmyra. On January 12th,
2017, the Syrian Army's 5th Legion and Russian special forces launched a
counterattack known as the ``Palmyra offensive'', with the aim of
retaking several important hydrocarbon deposits including Hayyan.

The timing of well fires aligns closely with a detailed timeline of the
campaign.The Near Infrared Sentinel-2 image below shows the layout of
the Hayyan Gas Plant and the wells in the Hayyan gas field:

The Syrian Army took the Hayyan gas field on
\href{https://www.almasdarnews.com/article/syrian-army-liberates-hayyan-gas-fields-west-palmyra/}{February
4th}, and retreating ISIS fighters set fire to wells 1, and 3. However,
ISIS managed to briefly retake the Hayyan field on
\href{https://www.almasdarnews.com/article/isis-retakes-hayyan-gas-fields-new-bid-expand-west-palmyra/}{February
7th}, setting fire to wells 2 and 4. These moments in the Palmyra
Offensive are captured in NIR signatures

Interestingly, despite the massive explosion caused by the bombing of
the Hayyan Gas Plant, no prolonged thermal anomalies were detected over
the area of the plant itself. The well fires, on the other hand, lasted
for months. Below is an image of well fire at the Hayyan field taken
from this \href{https://www.youtube.com/watch?v=WFe9abYyqK0}{video};
based on the nearby infrastructure and date (04/02/2017) of posting, it
is likely Well-3.

\hypertarget{refinery-identification}{%
\chapter*{Refinery Identification}\label{refinery-identification}}
\addcontentsline{toc}{chapter}{Refinery Identification}

\markboth{Refinery Identification}{Refinery Identification}

\emph{Topics: multispectral satellite imagery, machine learning,
informal economies, war.}

In Syria, over a decade of war has ravaged one of its most important
industries. Oil is a basic necessity for local residents who need to
heat their homes and keep the lights on. It's also an important source
of income for armed groups who control production; by some estimates the
Islamic State was making over
\href{https://www.rand.org/blog/2017/10/oil-extortion-still-paying-off-for-isis.html}{\$40
million} per month in oil revenues, and the Syrian Democratic Forces
were making
\href{https://www.al-monitor.com/originals/2021/08/syrian-government-kurds-discuss-plans-oil-trade}{\$120
million} per year selling oil to their adversaries, the Syrian
Government. This, in turn, has made oil production facilies a frequent
target of
\href{https://www.gov.uk/government/publications/british-forces-air-strikes-in-iraq-monthly-list/january-2015}{airstrikes},
leading to catastrophic environmental consequences.

The destruction of Syria's oil infrastructure and its importance as a
source of revenue for armed groups has led to a massive rise in
makeshift oil extraction and refining. These makeshift refineries are
often constructed by digging a large pit, lining it with a tarp, and
filling it with polluted water. A furnace heats crude oil, which is run
through a pipe cooled by the basin and collected in drums:

\includegraphics{./images/makeshift-refining.png} Wim Zwijnenburg wrote
an excellent
\href{https://www.bellingcat.com/news/2020/04/24/dying-to-keep-warm-oil-trade-and-makeshift-refining-in-north-west-syria/}{Bellingcat
article} on the subject, which you should read before going any further
in this tutorial. In the article, Wim notes that these facilities ``can
be spotted by the ditch and the black spot at the end with oil waste
residues, which blacken the soil around the furnace.'' These refineries
also frequently leak, blackening large swaths of land around them.

\begin{figure}

{\centering \includegraphics{./images/refinery.png}

}

\caption{source:
https://www.bellingcat.com/news/2020/04/24/dying-to-keep-warm-oil-trade-and-makeshift-refining-in-north-west-syria/}

\end{figure}

Looking around Northwestern Syria, we can see agricultural fields
pockmarked by these makeshift refineries (you can pan around and zoom
in):

\begin{verbatim}
Map(center=[36.936622, 42.118185], controls=(ZoomControl(options=['position', 'zoom_in_text', 'zoom_in_title',…
\end{verbatim}

Previous efforts to quantify informal oil production have involved
manually sifting through satellite imagery and counting them. This is a
painstaking process that leaves a number of important questions
unanswered. Even if we were to count all of the individual refineries,
could we get an estimate of polluted area? What if we wanted to count
the refineries in a new part of Syria? Or get annual or even monthly
estimates of new refineries?

Below is an Earth Engine application that automates the detection of
makeshift refineries in Northeastern Syria, using mutlispectral
satellite imagery and machine learning. Blue dots represent the
locations of predicted makeshift oil refineries and general oil
pollution, while red areas indicate areas predicted to be contaminated
by oil.

\begin{figure*}

\end{figure*}

You can draw an Area of Interest (AOI) and get the total number of
contaminated points as well as the total number of contaminated square
meters within the AOI. drawing multiple AOIs will show a running total
of these statistics. It's not perfect-- it misses some refineries and
falsely identifies some others-- but it is generally quite accurate; you
can visually verify the results of the prediction by zooming in using
the ``+'' button. You can toggle different layers using the ``layers''
tab as well. This tool could be used to get an estimate of oil
production in a user-defined area, and eventually to direct cleanup
efforts. The fullscreen version of the application can be found
\href{https://ollielballinger.users.earthengine.app/view/rojavaoil}{here},
and the source code
\href{https://code.earthengine.google.com/7a80f10412e1eb2a4d2c5d95989e70bd}{here}.
This tutorial will first cover the basics of multispectral remote
sensing, before moving into a step-by-step guide in the construction of
this model.

\hypertarget{machine-learning-workflow}{%
\chapter*{Machine Learning Workflow}\label{machine-learning-workflow}}
\addcontentsline{toc}{chapter}{Machine Learning Workflow}

\markboth{Machine Learning Workflow}{Machine Learning Workflow}

\hypertarget{pre-processing-2}{%
\section*{Pre-Processing}\label{pre-processing-2}}
\addcontentsline{toc}{section}{Pre-Processing}

\markright{Pre-Processing}

As always, the first step in our project will be to load and pre-process
satellite imagery. For this project, we'll be using Sentinel-2 imagery.
Let's load imagery from 2020-2021, filter out cloudy images, and define
visualization parameters:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ start}\OperatorTok{=}\StringTok{\textquotesingle{}2020{-}04{-}01\textquotesingle{}}
\KeywordTok{var}\NormalTok{ end}\OperatorTok{=}\StringTok{\textquotesingle{}2021{-}07{-}01\textquotesingle{}}

\KeywordTok{var}\NormalTok{ bands }\OperatorTok{=}\NormalTok{ [}\StringTok{\textquotesingle{}B2\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B5\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B6\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B7\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B8A\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B11\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B12\textquotesingle{}}\NormalTok{]}

\KeywordTok{var}\NormalTok{ sentinel }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{\textquotesingle{}COPERNICUS/S2\_SR\textquotesingle{}}\NormalTok{)}
                  \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{date}\NormalTok{(start}\OperatorTok{,}\NormalTok{ end))}
                  \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{lt}\NormalTok{(}\StringTok{\textquotesingle{}CLOUDY\_PIXEL\_PERCENTAGE\textquotesingle{}}\OperatorTok{,} \DecValTok{10}\NormalTok{))}
                  \OperatorTok{.}\FunctionTok{mean}\NormalTok{()}
                  \OperatorTok{.}\FunctionTok{select}\NormalTok{(bands)}

\KeywordTok{var}\NormalTok{ s\_rgb }\OperatorTok{=}\NormalTok{ \{}
  \DataTypeTok{min}\OperatorTok{:} \FloatTok{0.0}\OperatorTok{,}
  \DataTypeTok{max}\OperatorTok{:} \DecValTok{3000}\OperatorTok{,}
  \DataTypeTok{bands}\OperatorTok{:}\NormalTok{[}\StringTok{\textquotesingle{}B4\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}B2\textquotesingle{}}\NormalTok{]}\OperatorTok{,}
  \DataTypeTok{opacity}\OperatorTok{:}\DecValTok{1}
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

When loading the Sentinel-2 imagery, I've also onlyh selected the bands
that we will ultimately use in our analysis. There are a number of other
bands included in the data that we don't need. I've omitted a few bands
(B1, B9, B10) because they're collected at a much lower spatial
resolution (60 meters) compared to the other bands.

A couple types of landcover are so readily identifiable that we can
remove them with thresholds. Water and vegetation both have spectral
indices; we looked at NDVI above, but there's a similar one for water
called NDWI. These can be calculated from Sentinel-2 imagery as follows:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ ndvi}\OperatorTok{=}\NormalTok{sentinel}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}B8\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B4\textquotesingle{}}\NormalTok{])}
                  \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}nd\textquotesingle{}}\NormalTok{]}\OperatorTok{,}\NormalTok{[}\StringTok{\textquotesingle{}ndvi\textquotesingle{}}\NormalTok{])}

\KeywordTok{var}\NormalTok{ ndwi}\OperatorTok{=}\NormalTok{sentinel}\OperatorTok{.}\FunctionTok{normalizedDifference}\NormalTok{([}\StringTok{\textquotesingle{}B3\textquotesingle{}}\OperatorTok{,}\StringTok{\textquotesingle{}B8\textquotesingle{}}\NormalTok{])}
                  \OperatorTok{.}\FunctionTok{select}\NormalTok{([}\StringTok{\textquotesingle{}nd\textquotesingle{}}\NormalTok{]}\OperatorTok{,}\NormalTok{[}\StringTok{\textquotesingle{}ndwi\textquotesingle{}}\NormalTok{])}
\end{Highlighting}
\end{Shaded}

We use the \texttt{normalizedDifference} function and specify which
bands we want to use for each index. NDVI uses the red and near infrared
bands (B4 and B8), while NDWI uses bands 3 and 8. Finally, we want to
rename the resulting band from `nd' to the name of the spectral index.

Now we can use these indices to filter out water and vegetation. We do
this using the \texttt{updateMask} function, and specify that we want to
remove areas that have an NDVI value higher than 0.2 and and NDWI value
higher than 0.3. You can play around with these thesholds until you
achieve the desired results.

\begin{Shaded}
\begin{Highlighting}[]

\KeywordTok{var}\NormalTok{ image}\OperatorTok{=}\NormalTok{sentinel}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(ndwi}\OperatorTok{.}\FunctionTok{lt}\NormalTok{(}\FloatTok{0.3}\NormalTok{))}
                  \OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(ndvi}\OperatorTok{.}\FunctionTok{lt}\NormalTok{(}\FloatTok{0.2}\NormalTok{))}
                  \OperatorTok{.}\FunctionTok{addBands}\NormalTok{(ndvi)}
                  \OperatorTok{.}\FunctionTok{select}\NormalTok{(bands)}
\end{Highlighting}
\end{Shaded}

We also want to only select bands that are relevant to our analysis;
Sentinel

Finally, let's clip the image to our Area of Interest (AOI) and add it
to the map using the visualization parameters we defined earlier.

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(image}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(AOI)}\OperatorTok{,}\NormalTok{ s\_rgb}\OperatorTok{,} \StringTok{\textquotesingle{}Sentinel\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{figure}

{\centering \includegraphics{./images/rojava_preprocessed.png}

}

\caption{water and vegetation have been removed from this Sentinel-2
image. What remains is largely fallow agricultural land, urban areas,
and oil spills.}

\end{figure}

Now that we've loaded and preporcessed our satellite imagery, we can
proceed with the rest of our task. Ultimately, we want to create a map
of the study area which shows us different ``landcovers'' (materials).
This can broadly be achieved in three steps:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Generate labeled landcover data
\item
  Train a model using labeled data
\item
  Validate the model
\end{enumerate}

\hypertarget{generating-labeled-data}{%
\section*{1. Generating Labeled Data}\label{generating-labeled-data}}
\addcontentsline{toc}{section}{1. Generating Labeled Data}

\markright{1. Generating Labeled Data}

A vital step in any machine learning workflow is the generation of
labeled data, which we will use to train a model to differentiated
between different types of land cover and later to test the model's
accuracy. By looking around the study area, we can get a sense of the
different land cover classes that we might encounter:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Agricultural Land
\item
  Urban Areas
\item
  Oil Contamination
\end{enumerate}

Naturally we could subdivide each of these into sub-categories, and
there are probably other classes we haven't included that may be present
in the study area. The choice of classes is partly informed by the
nature of the task at hand. In theory, the most efficient number of
classes for this task would be two: oil, and everything else. The
problem is that the ``everything else'' category would be pretty noisy
since it would include a wide range of materials, making it harder to
distinguish this from oil. In practice, a visual inspection of major
landcover classes in the study area is a quick-and-dirty way of getting
at roughly the right number of classes. This is also an iterative
process: you can start with a set of labeled data, look at the model
results, and adjust your sampling accordingly. More on this later.

The main landcover class we're interested in is, of course, oil. Some
oil contamination is readily visible from the high resolution satellite
basemap; rivers of oil flow from the leaking
\href{https://zoom.earth/\#view=36.947921,42.02871,16z/overlays=heat,labels:off,crosshair}{Ger
Zero refinery}. We can draw polygons around the oil contamination like
so:

\includegraphics{./images/ger_zero.png}

The same process is applied to agricultural land and urban areas. In
general, you want to make sure that you're sampling from all across the
study area. I've generated between 4-10 polygons per landcover class in
different places. We're now left with a featureCollection composed of
polygons for each class. I've named them \texttt{oil},
\texttt{agriculture}, and \texttt{urban}.

However, I don't just want to use all of the pixels contained in these
polygons for training. There are several reasons for this. First, it
would likely lead to
\href{https://en.wikipedia.org/wiki/Overfitting}{overfitting}. Second,
there are probably over a million pixels between all of the polygons,
which would slow things down unnecessarily. Third, I haven't drawn the
polygons to be equal sizes across classes, so I could end up with way
more points from one class compared to another. It's OK to have some
imbalance between classes, but you don't want it to be extreme.

As such, the next step involves taking random samples of points from
\emph{within} these polygons. I do so using the \texttt{randomPoints}
function:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ oil\_points}\OperatorTok{=}\NormalTok{ee}\OperatorTok{.}\AttributeTok{FeatureCollection}\OperatorTok{.}\FunctionTok{randomPoints}\NormalTok{(oil}\OperatorTok{,} \DecValTok{3000}\NormalTok{)}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(i)\{}
  \ControlFlowTok{return}\NormalTok{ i}\OperatorTok{.}\FunctionTok{set}\NormalTok{(\{}\StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{:} \DecValTok{0}\NormalTok{\})\})}
  
\KeywordTok{var}\NormalTok{ urban\_points}\OperatorTok{=}\NormalTok{ee}\OperatorTok{.}\AttributeTok{FeatureCollection}\OperatorTok{.}\FunctionTok{randomPoints}\NormalTok{(urban}\OperatorTok{,} \DecValTok{1000}\NormalTok{)}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(i)\{}
  \ControlFlowTok{return}\NormalTok{ i}\OperatorTok{.}\FunctionTok{set}\NormalTok{(\{}\StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{:} \DecValTok{1}\NormalTok{\})\})}
  
\KeywordTok{var}\NormalTok{ agriculture\_points}\OperatorTok{=}\NormalTok{ee}\OperatorTok{.}\AttributeTok{FeatureCollection}\OperatorTok{.}\FunctionTok{randomPoints}\NormalTok{(agriculture}\OperatorTok{,} \DecValTok{2000}\NormalTok{)}\OperatorTok{.}\FunctionTok{map}\NormalTok{(}\KeywordTok{function}\NormalTok{(i)\{}
  \ControlFlowTok{return}\NormalTok{ i}\OperatorTok{.}\FunctionTok{set}\NormalTok{(\{}\StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{:} \DecValTok{2}\NormalTok{\})\})}
\end{Highlighting}
\end{Shaded}

In the first line, I create a new featureCollection called
\texttt{oil\_points} which contains 3000 points sampled from the
polygons in the \texttt{oil} featureCollection. I then map through each
of these points, and set a property called ``class'' equal to 0. I do
the same for the urban and agricultural areas, setting the ``class''
property of these featureCollections to 1 and 2, respectively.
Ultimately, our model will output a raster in which each pixel will
contain one of these three values. A value of 0 in the output will
represent the model predicting that that pixel is oil, based on the
training data; a value of 1 would indicate predicted urban land cover,
and 2 predicted agricultural landcover.

Now we want to create one feature collection called ``sample'', which
will contain all three sets of points.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ sample}\OperatorTok{=}\NormalTok{ee}\OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{([oil\_points}\OperatorTok{,}
\NormalTok{                                  urban\_points}\OperatorTok{,}
\NormalTok{                                  agriculture\_points}
\NormalTok{                                  ])}
                                  \OperatorTok{.}\FunctionTok{flatten}\NormalTok{()}
                                  \OperatorTok{.}\FunctionTok{randomColumn}\NormalTok{()}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

We've also assigned a property called ``random'' using the
\texttt{randomColumn} function. This lets us split our featureCollection
into two: one used for training the model, and one used for validation.
We'll use a 70-30 split.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ split}\OperatorTok{=}\FloatTok{0.7}
\KeywordTok{var}\NormalTok{ training\_sample }\OperatorTok{=}\NormalTok{ sample}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{lt}\NormalTok{(}\StringTok{\textquotesingle{}random\textquotesingle{}}\OperatorTok{,}\NormalTok{ split))}\OperatorTok{;}
\KeywordTok{var}\NormalTok{ validation\_sample }\OperatorTok{=}\NormalTok{ sample}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{gte}\NormalTok{(}\StringTok{\textquotesingle{}random\textquotesingle{}}\OperatorTok{,}\NormalTok{ split))}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\hypertarget{training-a-model}{%
\section*{2. Training a Model}\label{training-a-model}}
\addcontentsline{toc}{section}{2. Training a Model}

\markright{2. Training a Model}

Having generated labeled training and testing data, we now want to teach
an algorithm to associate the pixels in those areas (in particular,
their spectral profiles) with a specific landcover class.

The list of points we generated in the previous step contain a label (0:
oil, 1: urban, 2: agriculture). However, they do not yet contain any
information about the spectral profile of the Sentinel-2 image. The
\texttt{sampleRegions} function lets us assign a the band values from an
image as properties to our feature collection. We do this for both
training sample and the validation sample.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ training }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{sampleRegions}\NormalTok{(\{}
  \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ training\_sample}\OperatorTok{,}
  \DataTypeTok{properties}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}class\textquotesingle{}}\NormalTok{]}\OperatorTok{,}
  \DataTypeTok{scale}\OperatorTok{:} \DecValTok{10}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}

\KeywordTok{var}\NormalTok{ validation }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{sampleRegions}\NormalTok{(\{}
  \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ validation\_sample}\OperatorTok{,}
  \DataTypeTok{properties}\OperatorTok{:}\NormalTok{ [}\StringTok{\textquotesingle{}class\textquotesingle{}}\NormalTok{]}\OperatorTok{,}
  \DataTypeTok{scale}\OperatorTok{:} \DecValTok{10}
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Each point in the featureCollections above will contain a property
denoting each Sentinel-2 band's value at that location, as well as the
property denoting the class label.

Now we're ready to train the model. We'll be using a
\href{https://en.wikipedia.org/wiki/Random_forest}{Random Forest}
classifier, which basically works by trying to separate your data into
the specified classes by setting lots of thresholds in your input
properties (in our case, Sentinel-2 band values). It's a versatile and
widely-used model.

We first call a random forest classifier with 500 trees. More trees
usually yields higher accuracy, though there are diminishing returns.
Too many trees will result in your computation timing out. We then train
the model using the \texttt{train} function, which we supply with the
training data as well as the name of the property that contains our
class labels (``class'').

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ model }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Classifier}\OperatorTok{.}\FunctionTok{smileRandomForest}\NormalTok{(}\DecValTok{500}\NormalTok{)}
                          \OperatorTok{.}\FunctionTok{train}\NormalTok{(training}\OperatorTok{,} \StringTok{\textquotesingle{}class\textquotesingle{}}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The trained model now associates Sentinel-2 band values with one of
three landcover classes. We can now feed the model pixels it has never
seen before, and it will use what it now knows about the spectral
profiles of the differnt classes to predict the class of the new pixel.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ prediction }\OperatorTok{=}\NormalTok{ image}\OperatorTok{.}\FunctionTok{classify}\NormalTok{(model)}
\end{Highlighting}
\end{Shaded}

\texttt{prediction} is now a raster which contains one of three values
(0: oil, 1: urban, 2: agriculture). We're only interested in oil, so
let's isolate the regions in this raster that have a value of 0, and add
them in red to the map:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ oil\_prediction}\OperatorTok{=}\NormalTok{prediction}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(prediction}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\DecValTok{0}\NormalTok{))}

\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(oil\_prediction}\OperatorTok{,}\NormalTok{ \{}\DataTypeTok{palette}\OperatorTok{:}\StringTok{\textquotesingle{}red\textquotesingle{}}\NormalTok{\}}\OperatorTok{,} \StringTok{\textquotesingle{}Predicted Oil Conamination\textquotesingle{}}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

\includegraphics{./images/ger_zero_pred.png}

\hypertarget{validation}{%
\section*{3. Validation}\label{validation}}
\addcontentsline{toc}{section}{3. Validation}

\markright{3. Validation}

The image above should look somewhat familiar. It's Ger Zero, where we
trained part of our model. We can see in red the areas which the model
predicts to be oil pollution. These largley align with the areas that we
can see as being contaminated based on the high resolution basemap. It's
not perfect, but it's pretty good.

Let's scroll to another area, far from where the model was trained.
\includegraphics{./images/small_refinery.png} This image shows two
clusters of makeshift refineries which were identified by the model.
This is good, though we can only get so far by visually inspecting the
output from our model. To get a better sense of our model's performance,
we can use the validation data that we generated previously. Remember,
these are labeled points which our model was not trained on, and has
never seen before.

We'll take the \texttt{validation} featureCollection containing our
labeled points, and have our model classify it.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ validated }\OperatorTok{=}\NormalTok{ validation}\OperatorTok{.}\FunctionTok{classify}\NormalTok{(model)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now the \texttt{validated} variable is a featureCollection which
contains both manual labels and predicted labels from our model. We can
compare the manual labels to the predicted output to get a sense of how
well our model is performing. This is called a Confusion Matrix (or an
Error Matrix):

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ testAccuracy }\OperatorTok{=}\NormalTok{ validated}\OperatorTok{.}\FunctionTok{errorMatrix}\NormalTok{(}\StringTok{\textquotesingle{}class\textquotesingle{}}\OperatorTok{,} \StringTok{\textquotesingle{}classification\textquotesingle{}}\NormalTok{)}\OperatorTok{;}

\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Confusion Matrix \textquotesingle{}}\OperatorTok{,}\NormalTok{ testAccuracy)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\begin{longtable}[]{@{}
  >{\centering\arraybackslash}p{(\columnwidth - 8\tabcolsep) * \real{0.2059}}
  >{\centering\arraybackslash}p{(\columnwidth - 8\tabcolsep) * \real{0.2500}}
  >{\centering\arraybackslash}p{(\columnwidth - 8\tabcolsep) * \real{0.1324}}
  >{\centering\arraybackslash}p{(\columnwidth - 8\tabcolsep) * \real{0.1618}}
  >{\centering\arraybackslash}p{(\columnwidth - 8\tabcolsep) * \real{0.2500}}@{}}
\toprule()
\begin{minipage}[b]{\linewidth}\centering
\end{minipage} & \begin{minipage}[b]{\linewidth}\centering
\end{minipage} & \begin{minipage}[b]{\linewidth}\centering
\end{minipage} & \begin{minipage}[b]{\linewidth}\centering
\emph{Labels}
\end{minipage} & \begin{minipage}[b]{\linewidth}\centering
\end{minipage} \\
\midrule()
\endhead
& & \textbf{Oil} & \textbf{Urban} & \textbf{Agriculture} \\
& \textbf{Oil} & 876 & 1 & 5 \\
\emph{Prediction} & \textbf{Urban} & 0 & 168 & 8 \\
& \textbf{Agriculture} & 1 & 4 & 514 \\
\bottomrule()
\end{longtable}

Now, we can see that of the 877 points that were labeled ``oil'', only
one was falsely predicted to be agicultural land. The model also falsely
predicted as oil one point that was labeled urban, and five points that
were labeled agriculture. Not bad. We can get a sense of the model's
overall accuracy using the \texttt{accuracy} function on the confusion
matrix:

\begin{Shaded}
\begin{Highlighting}[]
\FunctionTok{print}\NormalTok{(}\StringTok{\textquotesingle{}Validation overall accuracy: \textquotesingle{}}\OperatorTok{,}\NormalTok{ testAccuracy}\OperatorTok{.}\FunctionTok{accuracy}\NormalTok{())}
\end{Highlighting}
\end{Shaded}

This tells us that the overall accuracy of our model is around 98\%.
However, we shouldn't take this estimate at face value. There are a
number of complicated reasons
(\href{https://www.sciencedirect.com/topics/computer-science/spatial-autocorrelation\#:~:text=Spatial\%20autocorrelation\%20is\%20the\%20term,together\%20to\%20have\%20similar\%20values.}{spatial
autocorrelation} in the training data, for example) why this figure is
probably inflatred. If we were submitting this analysis to a
peer-reviewed journal, we'd take great care in addressing this, but for
our purposes we can use the accuracy statistics to guide our analysis
and get a rough sense of how well the model is performing.

This model isn't perfect; it often misclassifies the shorelines of lakes
as oil, or certain parts of urban areas. As previously mentioned,
training a model is often an iterative process. At this stage, if your
accuracy is not as high as you'd like it to be, you can use the output
to figure out how to tweak the model. For example, you may observe that
your model is confusing urban areas with oil spills. You can draw a
polygon over the erroneous area, label it urban landcover and retrain
the model thereby hopefully improving accuracy. We could further refine
our model in this way.

\hypertarget{communicating-the-results}{%
\chapter*{Communicating the Results}\label{communicating-the-results}}
\addcontentsline{toc}{chapter}{Communicating the Results}

\markboth{Communicating the Results}{Communicating the Results}

Now that we've got a model that can identify oil from multispectral
satellite imagery fairly well, we can set about making our results
accessible.

One of the things we're particularly interested in is the distribution
of small refineries. The way we're currently visualizing the prediction
(the raster output from the model where predicted oil is shown in red
and everything else is transparent) makes it hard to see these small
refineries when we zoom out:

\includegraphics{./images/big_red.png}

We can convert our raster into a series of points using the
\texttt{reduceToVectors} function. In essence, this takes homogenous
regions of an image (e.g., an area predicted to be oil surrounded by an
area not predicted to be oil) and converts it into a point:

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ vectors }\OperatorTok{=}\NormalTok{ oil\_prediction}\OperatorTok{.}\FunctionTok{reduceToVectors}\NormalTok{(\{}
  \DataTypeTok{geometry}\OperatorTok{:}\NormalTok{ AOI}\OperatorTok{,}
  \DataTypeTok{scale}\OperatorTok{:} \DecValTok{10}\OperatorTok{,}
  \DataTypeTok{geometryType}\OperatorTok{:} \StringTok{\textquotesingle{}centroid\textquotesingle{}}\OperatorTok{,}
  \DataTypeTok{eightConnected}\OperatorTok{:} \KeywordTok{true}\OperatorTok{,}
  \DataTypeTok{labelProperty}\OperatorTok{:} \StringTok{\textquotesingle{}classification\textquotesingle{}}\OperatorTok{,}
  \DataTypeTok{maxPixels}\OperatorTok{:}\DecValTok{1653602926}
\NormalTok{  \})}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(AOI)}

\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(vectors}\OperatorTok{.}\FunctionTok{style}\NormalTok{(\{}\DataTypeTok{color}\OperatorTok{:} \StringTok{\textquotesingle{}black\textquotesingle{}}\OperatorTok{,} \DataTypeTok{fillColor}\OperatorTok{:} \StringTok{\textquotesingle{}\#00f2ff\textquotesingle{}}\OperatorTok{,} \DataTypeTok{pointSize}\OperatorTok{:}\DecValTok{5}\NormalTok{\})}\OperatorTok{,}\NormalTok{\{\}}\OperatorTok{,}\StringTok{\textquotesingle{}Oil Contamination Points\textquotesingle{}}\OperatorTok{,}\KeywordTok{false}\NormalTok{)}
\end{Highlighting}
\end{Shaded}

Now the distribution of small refineries is much more easily visible as
blue dots:

\includegraphics{./images/points.png}

If we zoom out even further, we can see clusters of points that
correspond to areas of high oil production. Using geolocated
photographs, we can roughly ground-truth the model output:

\includegraphics{./images/UNEP.PNG}

\hypertarget{ship-detection}{%
\chapter*{Ship Detection}\label{ship-detection}}
\addcontentsline{toc}{chapter}{Ship Detection}

\markboth{Ship Detection}{Ship Detection}

\hypertarget{introduction-21}{%
\chapter*{Introduction}\label{introduction-21}}
\addcontentsline{toc}{chapter}{Introduction}

\markboth{Introduction}{Introduction}

There's a huge amount of data available on the internet about ship
movements, most of which draw on the Automatic Identification System
(AIS) which is a system that uses radio to broadcast the identity,
position, course, speed, and other data about ships.
\href{https://www.marinetraffic.com/en/ais-api-services}{MarineTraffic},
for example, provides an API that allows you to query the location of
ships in real time as well as historical vessel tracks and lots of other
useful data. Unfortunately most sources of AIS data are paywalled, and
AIS can be turned off or manipulated to hide the identity or position of
the ship. In fact, most of the stuff we're interested in investigating
probably happens when AIS is turned off.

Though ships can hide by turning off their AIS transponders, they can't
hide from satellites. In this tutorial, we're going to build an
application that uses Synthetic Aperture Radar (SAR) from the European
Space Agency's Sentinel-1 satellite to automatically identify ships,
regardless of whether they've got their transponders turned on or off.
Here's the finished application:

\begin{figure*}

\end{figure*}

\hypertarget{how-it-works}{%
\section*{How it Works}\label{how-it-works}}
\addcontentsline{toc}{section}{How it Works}

\markright{How it Works}

The app has two main panels:

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  A control panel on the left that allows the user to interact with the
  application
\item
  A map on the right that displays the results
\end{enumerate}

The control panel has a date slider that allows the user to load imagery
from a particular year. Below that is a graph that shows the number of
ships detected over time within that year. A slider underneath the graph
lets us toggle the sensitivity of the ship detection process. Finally, a
button at the bottom lets the user draw their own area of interest on
the map, and the app will automatically detect ships within that area.

The map panel visualizes the results of the ship detection process and
has three layers. The bottom layer is the Sentinel-1 image that we're
using to detect ships; it's blue/purple, and if you zoom in and look
closley you can see bright specks in the sea, which are ships. When
Sentinel-1 sends a pulse of radio waves onto a flat surface like the
sea, there is very little to reflect the waves back to the satellite--
they just bounce off into space. A low return signal means we'll see a
darker color on our map. But when the raio waves hit a ship they are
reflected back to the satelite and generate a higher return signal, and
therefore a much brighter color. The second layer on the map displays a
bunch of green points; each one of these is a detected ship. The last
layer shows the red outline of the area of interest that the user drew
on the map. You can zoom in on the map by holding down the command
button and scrolling up and down.

When the application is first loaded it is centered on an area just
north of the Suez Canal, and is analyzing imagery from 2021. We can see
a bunch of green dots in the AOI, which is the main waiting area for
ships waiting to transit the canal. It's a bit crowded because it's
visualizing all of the ships detected in the entire year. We can display
imagery from a single day by clicking on a point in the graph on the
left, which you will notice displays a huge spike in the number of ships
detected around March.

You might remember that on March 23rd, 2021, the Ever Given-- a 400m
long container ship-- got stuck in the Suez Canal. The ship was blocking
the canal for six days, and it's estimated that it cost the global
economy \$400 million per day. If you click on the tip of the spike on
March 30th, you can see backup of around 150 ships waiting for the canal
to be cleared. You can also zoom in on a particular date range by
scrolling and dragging on the graph. If you zoom in on the spike, you
can then select imagery from early April to compare the number of ships
in the waiting area after the blockage was cleared. In normal times we
can see a regular pattern in the number of ships in the waiting area
ranging between 15 and 40 ships.

If you're closely zoomed in to the map and load imagery from different
days by clicking on the graph, you can compare the bright spots on the
Sentinel image and the green dots. The ship detection process is pretty
accurate, and we typically see one green dot per ship. However, you may
notice that we occasionally miss a ship. This is because the ship
detection process is based on a threshold, and if the ship is too small
it may not generate a high enough return signal to be detected. You can
increase the sensitivity of the ship detection process by moving the
slider below the graph. This will increase the number of ships detected,
but it may also increase the number of false positives.

The next section focuses on building this application. After that, we'll
have a look at a few different use cases for this sort of maritime
surveillance.

\hypertarget{building-the-application}{%
\chapter*{Building the Application}\label{building-the-application}}
\addcontentsline{toc}{chapter}{Building the Application}

\markboth{Building the Application}{Building the Application}

\hypertarget{setup}{%
\section*{Setup}\label{setup}}
\addcontentsline{toc}{section}{Setup}

\markright{Setup}

The first step is to configure the map and import the necessary
datasets. By default, we want the app to be centered on the Suez Canal.
Then, we want to import the Digital Surface Model (DSM) from the ALOS
World 3D-30 dataset. This dataset provides a 30m resolution elevation
model of the Earth which we will use to mask out the land. Finally, we
want to import the Sentinel 1 dataset. We will use the VV polarization
and the Interferometric Wide (IW) mode. We will also sort the images by
date.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Center the map on the Suez Canal and set map options}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\FloatTok{32.327}\OperatorTok{,} \FloatTok{31.4532}\OperatorTok{,} \DecValTok{10}\NormalTok{)}\OperatorTok{;}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{"Hybrid"}\NormalTok{)}\OperatorTok{;}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setControlVisibility}\NormalTok{(\{ }\DataTypeTok{all}\OperatorTok{:} \KeywordTok{false}\NormalTok{ \})}\OperatorTok{;}

\CommentTok{// Import the Digital Surface Model (DSM) from the ALOS World 3D{-}30 dataset}
\KeywordTok{var}\NormalTok{ dem }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{"JAXA/ALOS/AW3D30/V3\_2"}\NormalTok{)}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{"DSM"}\NormalTok{)}\OperatorTok{;}

\CommentTok{// Import the Sentinel 1 dataset}
\KeywordTok{var}\NormalTok{ s1 }\OperatorTok{=}\NormalTok{ ee}
  \OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{"COPERNICUS/S1\_GRD"}\NormalTok{)}
  \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{listContains}\NormalTok{(}\StringTok{"transmitterReceiverPolarisation"}\OperatorTok{,} \StringTok{"VV"}\NormalTok{))}
  \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{"instrumentMode"}\OperatorTok{,} \StringTok{"IW"}\NormalTok{))}
  \OperatorTok{.}\FunctionTok{sort}\NormalTok{(}\StringTok{"system:time\_start"}\NormalTok{)}\OperatorTok{;}

\CommentTok{// Define the default area of interest}
\KeywordTok{var}\NormalTok{ suez }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Polygon}\NormalTok{([}
\NormalTok{  [}
\NormalTok{    [}\FloatTok{32.17388584692775}\OperatorTok{,} \FloatTok{31.59541178442045}\NormalTok{]}\OperatorTok{,}
\NormalTok{    [}\FloatTok{32.17388584692775}\OperatorTok{,} \FloatTok{31.327159861902278}\NormalTok{]}\OperatorTok{,}
\NormalTok{    [}\FloatTok{32.4787564523965}\OperatorTok{,} \FloatTok{31.327159861902278}\NormalTok{]}\OperatorTok{,}
\NormalTok{    [}\FloatTok{32.4787564523965}\OperatorTok{,} \FloatTok{31.59541178442045}\NormalTok{]}\OperatorTok{,}
\NormalTok{  ]}\OperatorTok{,}
\NormalTok{])}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Now that we've gotten that out of the way, we can move on to the actual
detection of ships.

\hypertarget{ship-detection-1}{%
\section*{Ship Detection}\label{ship-detection-1}}
\addcontentsline{toc}{section}{Ship Detection}

\markright{Ship Detection}

You might expect the automatic identification of ships based on
synthetic aperture radar satellite imagery to involve a complex machine
learning algorithm or artificial intelligence. In fact, it can be done
in one line of code which sets a cutoff. If the return signal is greater
than 0, then we have a ship. If it's less than 0, then we don't. Simple
as that.

The main analytical function responsible for ship identification is the
\texttt{getVectors} function shown below. It takes an image as an input
and returns a FeatureCollection of points, each corresponding to a ship.
The function clips the image to the area of interest, selects the VV
polarization, and finally filters out areas where the VV value is
smaller than 0. This results in a raster image where the sea is black
and the ships are white. We then use the \texttt{reduceToVectors}
function to convert the raster image to a FeatureCollection of points.
The function returns this FeatureCollection, and sets a property called
\texttt{count} which is the number of ships detected in the image.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{function} \FunctionTok{getVectors}\NormalTok{(img) \{}
  \CommentTok{// Get the area of interest from the drawing tools widget.  }
  \KeywordTok{var}\NormalTok{ aoi }\OperatorTok{=}\NormalTok{ drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{.}\FunctionTok{getEeObject}\NormalTok{()}\OperatorTok{;}

  \CommentTok{// Clip the image to the area of interest}
  \CommentTok{// Select the VV polarization }
  \CommentTok{// Filter areas where the VV value is greater than 0}
  \KeywordTok{var}\NormalTok{ cutoff }\OperatorTok{=}\NormalTok{ img}\OperatorTok{.}\FunctionTok{clip}\NormalTok{(aoi)}\OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{"VV"}\NormalTok{)}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{0}\NormalTok{)}

  \CommentTok{// Convert the raster image to a FeatureCollection of points}
  \KeywordTok{var}\NormalTok{ points }\OperatorTok{=}\NormalTok{ cutoff}\OperatorTok{.}\FunctionTok{reduceToVectors}\NormalTok{(\{}
    \DataTypeTok{geometry}\OperatorTok{:}\NormalTok{ aoi}\OperatorTok{,}
    \DataTypeTok{scale}\OperatorTok{:}\NormalTok{ scaleSlider}\OperatorTok{.}\FunctionTok{getValue}\NormalTok{()}\OperatorTok{,}
    \DataTypeTok{geometryType}\OperatorTok{:} \StringTok{"centroid"}\OperatorTok{,}
    \DataTypeTok{eightConnected}\OperatorTok{:} \KeywordTok{true}\OperatorTok{,}
    \DataTypeTok{maxPixels}\OperatorTok{:} \DecValTok{1653602926}\OperatorTok{,}
\NormalTok{  \})}\OperatorTok{;}

  \CommentTok{// Set the number of ships detected in the image as a property called "count"}
  \KeywordTok{var}\NormalTok{ count }\OperatorTok{=}\NormalTok{ points}\OperatorTok{.}\FunctionTok{size}\NormalTok{()}\OperatorTok{;}
  \CommentTok{// Set the date of the image as a property called "system:time\_start"}
  \KeywordTok{var}\NormalTok{ date }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(img}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\StringTok{"system:time\_start"}\NormalTok{))}\OperatorTok{;}
  \ControlFlowTok{return}\NormalTok{ points}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{"count"}\OperatorTok{,}\NormalTok{ count)}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\StringTok{"system:time\_start"}\OperatorTok{,}\NormalTok{ date)}\OperatorTok{;}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

The \texttt{count} and \texttt{system:time\_start} properties are used
to create the graph of daily ship counts and allow the resulting vector
(point) data to interact with the date slider widget. An important
detail here is that the ``scale'' parameter of the
\texttt{reduceToVectors} function is set to the value of the scale
slider widget. This allows the user to adjust the resolution of the ship
detection process; a smaller value will allow us to detect smaller
ships.

\hypertarget{visualization}{%
\section*{Visualization}\label{visualization}}
\addcontentsline{toc}{section}{Visualization}

\markright{Visualization}

The \texttt{viz} function is responsible for displaying the results of
the ship detection process. It takes the area of interest, the vector
data, and the Sentinel 1 image as inputs. Nothing super complicated
here; we're just creating three layers and adding them to the map in
order: the underlying Sentinel-1 image raster, the ship vector data in
green, and the area of interest outline in red. We're using the
\texttt{Map.layers().set()} function to replace the existing layers with
the new ones, rather than addine new ones each time.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{function} \FunctionTok{viz}\NormalTok{(aoi}\OperatorTok{,}\NormalTok{ vectors}\OperatorTok{,}\NormalTok{ s1Filtered) \{}
  \CommentTok{// Create an empty image into which to paint the features, cast to byte.}
  \KeywordTok{var}\NormalTok{ empty }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{;}

  \CommentTok{// Paint all the polygon edges with the same number and width, display.}
  \KeywordTok{var}\NormalTok{ outline }\OperatorTok{=}\NormalTok{ empty}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{}
    \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ aoi}\OperatorTok{,}
    \DataTypeTok{color}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}
    \DataTypeTok{width}\OperatorTok{:} \DecValTok{3}\OperatorTok{,}
\NormalTok{  \})}\OperatorTok{;}

  \CommentTok{// Create a layer for the area of interest in red}
  \KeywordTok{var}\NormalTok{ aoi\_layer }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Map}\OperatorTok{.}\FunctionTok{Layer}\NormalTok{(outline}\OperatorTok{,}\NormalTok{ \{ }\DataTypeTok{palette}\OperatorTok{:} \StringTok{"red"}\NormalTok{ \}}\OperatorTok{,} \StringTok{"AOI"}\NormalTok{)}\OperatorTok{;}

  \CommentTok{// Create a layer for the vector data in green}
  \KeywordTok{var}\NormalTok{ vectorLayer }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Map}\OperatorTok{.}\FunctionTok{Layer}\NormalTok{(}
\NormalTok{    vectors}\OperatorTok{.}\FunctionTok{flatten}\NormalTok{()}\OperatorTok{,}
\NormalTok{    \{ }\DataTypeTok{color}\OperatorTok{:} \StringTok{"\#39ff14"}\NormalTok{ \}}\OperatorTok{,}
    \StringTok{"Vectors"}
\NormalTok{  )}\OperatorTok{;}

  \CommentTok{// Create a layer for the Sentinel 1 image in false color}
  \KeywordTok{var}\NormalTok{ sarLayer }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Map}\OperatorTok{.}\FunctionTok{Layer}\NormalTok{(}
\NormalTok{    s1Filtered}\OperatorTok{,}
\NormalTok{    \{ }\DataTypeTok{min}\OperatorTok{:}\NormalTok{ [}\OperatorTok{{-}}\DecValTok{25}\OperatorTok{,} \OperatorTok{{-}}\DecValTok{20}\OperatorTok{,} \OperatorTok{{-}}\DecValTok{25}\NormalTok{]}\OperatorTok{,} \DataTypeTok{max}\OperatorTok{:}\NormalTok{ [}\DecValTok{0}\OperatorTok{,} \DecValTok{10}\OperatorTok{,} \DecValTok{0}\NormalTok{]}\OperatorTok{,} \DataTypeTok{opacity}\OperatorTok{:} \FloatTok{0.8}\NormalTok{ \}}\OperatorTok{,}
    \StringTok{"SAR"}
\NormalTok{  )}\OperatorTok{;}

  \CommentTok{// Add the layers in order}
  \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\DecValTok{0}\OperatorTok{,}\NormalTok{ sarLayer)}\OperatorTok{;}
  \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\DecValTok{1}\OperatorTok{,}\NormalTok{ vectorLayer)}\OperatorTok{;}
  \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\DecValTok{2}\OperatorTok{,}\NormalTok{ aoi\_layer)}\OperatorTok{;}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

We want a function to handle the visualization because there are two
different situations in which we're going to visualize results, and we
dont want to repeat our code. The first situation is when the user draws
a new area of interest, moves the date slider, or alters the scale. In
this case, we want to visualize the results of the ship detection
process for the entire year's worth of Sentinel-1 imagery. The second
situation is when the user clicks on the chart to analyze a particular
day. In this case, we obviously only want to visualize the results of
the ship detection process on that day. With this function, we can
simply pass the appropriately filtered versions of the Sentinel-1 image
and vector data to the function, and it will visualize the results,
rather than having to write the same code twice.

\hypertarget{putting-it-all-together}{%
\section*{Putting it all together}\label{putting-it-all-together}}
\addcontentsline{toc}{section}{Putting it all together}

\markright{Putting it all together}

Having defined a few helper functions to handle the visualization and
ship detection process, we can now move on to the main function that
will perform the analysis. This will be performed by the
\texttt{daterangeVectors} function. In a nutshell, it read the user
specified date range from the date slider widget, and filter the
Sentinel 1 dataset to only include images within that period. Then, it
will loop through each Sentinel-1 image from that year and apply the
\texttt{getVectors} function to count the number of ships that fall
within the area of interest and generate a dataset of points
corresponding to detected ships. We'll then use the \texttt{viz}
function we just defined to visualize the all of the ship detections and
Sentinel-1 images in the AOI during that year stacked on top of each
other. Finally, we'll create a chart based on the number of ships
detected per day, and allow the user to click on the chart to visualize
the results for a particular day.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ daterangeVectors }\OperatorTok{=} \KeywordTok{function}\NormalTok{ () \{}
  
  \CommentTok{// Get the date range from the date slider widget.}
  \KeywordTok{var}\NormalTok{ range }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{DateRange}\NormalTok{(}
\NormalTok{    ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(dateSlider}\OperatorTok{.}\FunctionTok{getValue}\NormalTok{()[}\DecValTok{0}\NormalTok{])}\OperatorTok{,}
\NormalTok{    ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(dateSlider}\OperatorTok{.}\FunctionTok{getValue}\NormalTok{()[}\DecValTok{1}\NormalTok{])}
\NormalTok{  )}\OperatorTok{;}

  \CommentTok{// Get the area of interest from the drawing tools widget.}
  \KeywordTok{var}\NormalTok{ aoi }\OperatorTok{=}\NormalTok{ drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{.}\FunctionTok{getEeObject}\NormalTok{()}\OperatorTok{;}

  \CommentTok{// Hide the user{-}drawn shape.}
\NormalTok{  drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{.}\FunctionTok{setShown}\NormalTok{(}\KeywordTok{false}\NormalTok{)}\OperatorTok{;}

  \CommentTok{// Filter the Sentinel 1 dataset to only include images within the date range, and within the area of interest.}
  \KeywordTok{var}\NormalTok{ s1Filtered }\OperatorTok{=}\NormalTok{ s1}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(range}\OperatorTok{.}\FunctionTok{start}\NormalTok{()}\OperatorTok{,}\NormalTok{ range}\OperatorTok{.}\FunctionTok{end}\NormalTok{())}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(aoi)}\OperatorTok{;}
  
  \CommentTok{// Count the number of ships in each image using the getVectors function}
  \KeywordTok{var}\NormalTok{ vectors }\OperatorTok{=}\NormalTok{ s1Filtered}\OperatorTok{.}\FunctionTok{map}\NormalTok{(getVectors)}\OperatorTok{;}

  \CommentTok{// Use the viz function to visualize the results }
  \FunctionTok{viz}\NormalTok{(aoi}\OperatorTok{,}\NormalTok{ vectors}\OperatorTok{,}\NormalTok{ s1Filtered}\OperatorTok{.}\FunctionTok{max}\NormalTok{()}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(dem}\OperatorTok{.}\FunctionTok{lte}\NormalTok{(}\DecValTok{0}\NormalTok{)))}\OperatorTok{;}

  \CommentTok{// Create a chart of the number of ships per day}
  \KeywordTok{var}\NormalTok{ chart }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Chart}\OperatorTok{.}\AttributeTok{feature}
    \OperatorTok{.}\FunctionTok{byFeature}\NormalTok{(\{}
      \DataTypeTok{features}\OperatorTok{:}\NormalTok{ vectors}\OperatorTok{,}
      \DataTypeTok{xProperty}\OperatorTok{:} \StringTok{"system:time\_start"}\OperatorTok{,}
      \DataTypeTok{yProperties}\OperatorTok{:}\NormalTok{ [}\StringTok{"count"}\NormalTok{]}\OperatorTok{,}
\NormalTok{    \})}
    \OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(\{}
      \DataTypeTok{title}\OperatorTok{:} \StringTok{"Daily Number of Ships in Area of Interest"}\OperatorTok{,}
      \DataTypeTok{vAxis}\OperatorTok{:}\NormalTok{ \{ }\DataTypeTok{title}\OperatorTok{:} \StringTok{"Ship Count"}\NormalTok{ \}}\OperatorTok{,}
      \DataTypeTok{explorer}\OperatorTok{:}\NormalTok{ \{ }\DataTypeTok{axis}\OperatorTok{:} \StringTok{"horizontal"}\NormalTok{ \}}\OperatorTok{,}
      \DataTypeTok{lineWidth}\OperatorTok{:} \DecValTok{2}\OperatorTok{,}
      \DataTypeTok{series}\OperatorTok{:} \StringTok{"Area of Interest"}\OperatorTok{,}
\NormalTok{    \})}\OperatorTok{;}

  \CommentTok{// Add the chart at a fixed position, so that new charts overwrite older ones.}
\NormalTok{  controlPanel}\OperatorTok{.}\FunctionTok{widgets}\NormalTok{()}\OperatorTok{.}\FunctionTok{set}\NormalTok{(}\DecValTok{4}\OperatorTok{,}\NormalTok{ chart)}\OperatorTok{;}

  \CommentTok{// Add a click handler to the chart to filter the map by day.}
\NormalTok{  chart}\OperatorTok{.}\FunctionTok{onClick}\NormalTok{(filterDay)}\OperatorTok{;}
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

There's one function referenced above-- \texttt{filterDay}-- that we
haven't defined yet. This function is called when the user clicks on the
chart to analyze a particular day. It takes the date of the clicked day
as an input, filters the Sentinel-1 dataset and vector data accordingly,
and uses the \texttt{viz} function to display the results for that day.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{function} \FunctionTok{filterDay}\NormalTok{ (callback) \{}

    \CommentTok{// Get the date of the clicked day}
    \KeywordTok{var}\NormalTok{ date }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(callback)}\OperatorTok{;}

    \CommentTok{// Filter the vector data to only include images from that day}
    \KeywordTok{var}\NormalTok{ vectorDay }\OperatorTok{=}\NormalTok{ vectors}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(date)}\OperatorTok{;}
    
    \CommentTok{// Filter the Sentinel{-}1 imagery to only include images from that day}
    \KeywordTok{var}\NormalTok{ s1Day }\OperatorTok{=}\NormalTok{ s1}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(date)}\OperatorTok{.}\FunctionTok{max}\NormalTok{()}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(dem}\OperatorTok{.}\FunctionTok{lte}\NormalTok{(}\DecValTok{0}\NormalTok{))}\OperatorTok{;}

    \CommentTok{// Use the viz function to visualize the results}
    \FunctionTok{viz}\NormalTok{(aoi}\OperatorTok{,}\NormalTok{ vectorDay}\OperatorTok{,}\NormalTok{ s1Day)}\OperatorTok{;}
\NormalTok{\}}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The analytical portion of the application is now complete. Now we have
to build a user interface that lets us interact with the application.

\hypertarget{building-a-user-interface}{%
\section*{Building a User Interface}\label{building-a-user-interface}}
\addcontentsline{toc}{section}{Building a User Interface}

\markright{Building a User Interface}

There are four main steps in the process of creating the User Interface
(UI):

\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\tightlist
\item
  Configure the drawing tools that allow the user to draw a polygon on
  the map.
\item
  Create some widgets
\end{enumerate}

\hypertarget{drawing-tools}{%
\subsection*{Drawing Tools}\label{drawing-tools}}
\addcontentsline{toc}{subsection}{Drawing Tools}

We eventually want to allow the user to draw a polygon on the map, and
count the number of ships that fall within it. In order to do so, we
need to set up a few functions related to the drawing tools that allow
the user to do this. Among other things, we want to make sure that we're
clearing the old geometries so that we're only ever conducting analysis
inside the most recent user-drawn polygon, so we'll need to clear the
old ones. We also want to specify the type of polygon the user can draw,
which for ease will be a rectangle (you could change this to the actual
``polygon'' type if you wanted to draw more complex geometries).

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ drawingTools }\OperatorTok{=} \BuiltInTok{Map}\OperatorTok{.}\FunctionTok{drawingTools}\NormalTok{()}\OperatorTok{;}

\CommentTok{// Remove any existing layers}
\ControlFlowTok{while}\NormalTok{ (drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{length}\NormalTok{() }\OperatorTok{\textgreater{}} \DecValTok{0}\NormalTok{) \{}
  \KeywordTok{var}\NormalTok{ layer }\OperatorTok{=}\NormalTok{ drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{;}
\NormalTok{  drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{remove}\NormalTok{(layer)}\OperatorTok{;}
\NormalTok{\}}

\CommentTok{// Add a dummy layer to the drawing tools object (the Suez Canal box)}
\KeywordTok{var}\NormalTok{ dummyGeometry }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Map}\OperatorTok{.}\FunctionTok{GeometryLayer}\NormalTok{(\{}
  \DataTypeTok{geometries}\OperatorTok{:} \KeywordTok{null}\OperatorTok{,}
\NormalTok{\})}
  \OperatorTok{.}\FunctionTok{fromGeometry}\NormalTok{(suez)}
  \OperatorTok{.}\FunctionTok{setShown}\NormalTok{(}\KeywordTok{false}\NormalTok{)}\OperatorTok{;}

\CommentTok{// Add the dummy layer to the drawing tools object}
\NormalTok{drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{.}\FunctionTok{add}\NormalTok{(dummyGeometry)}\OperatorTok{;}


\CommentTok{// Create a function that clears existing geometries and lets the user draw a rectangle}
\KeywordTok{function} \FunctionTok{drawPolygon}\NormalTok{() \{}
  \KeywordTok{var}\NormalTok{ layers }\OperatorTok{=}\NormalTok{ drawingTools}\OperatorTok{.}\FunctionTok{layers}\NormalTok{()}\OperatorTok{;}
\NormalTok{  layers}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{.}\FunctionTok{geometries}\NormalTok{()}\OperatorTok{.}\FunctionTok{remove}\NormalTok{(layers}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{)}\OperatorTok{.}\FunctionTok{geometries}\NormalTok{()}\OperatorTok{.}\FunctionTok{get}\NormalTok{(}\DecValTok{0}\NormalTok{))}\OperatorTok{;}
\NormalTok{  drawingTools}\OperatorTok{.}\FunctionTok{setShape}\NormalTok{(}\StringTok{"rectangle"}\NormalTok{)}\OperatorTok{;}
\NormalTok{  drawingTools}\OperatorTok{.}\FunctionTok{draw}\NormalTok{()}\OperatorTok{;}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

\hypertarget{widgets}{%
\subsection*{Widgets}\label{widgets}}
\addcontentsline{toc}{subsection}{Widgets}

The control panel will eventually contain a few different widgets that
allow the user to interact with the application. We'll start by creating
a button that allows the user to draw a polygon on the map. We'll also
create a slider that allows the user to adjust the size of the ships
that are detected (remember, this manipualtes the ``scale'' parameter in
the \texttt{reduceToVectors} function used in the detection process).
The slider will have an accompanying label that tells the user what it
does.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a button that allows the user to draw a polygon on the map}
\KeywordTok{var}\NormalTok{ drawButton }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\FunctionTok{Button}\NormalTok{(\{}
  \DataTypeTok{label}\OperatorTok{:} \StringTok{"🔺"} \OperatorTok{+} \StringTok{" Draw a Polygon"}\OperatorTok{,}
  \DataTypeTok{onClick}\OperatorTok{:}\NormalTok{ drawPolygon}\OperatorTok{,}
  \DataTypeTok{style}\OperatorTok{:}\NormalTok{ \{ }\DataTypeTok{stretch}\OperatorTok{:} \StringTok{"horizontal"}\NormalTok{ \}}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}

\CommentTok{// Create a slider that allows the user to adjust the size of the ships that are detected}
\KeywordTok{var}\NormalTok{ scaleSlider }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\FunctionTok{Slider}\NormalTok{(\{}
  \DataTypeTok{min}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}
  \DataTypeTok{max}\OperatorTok{:} \DecValTok{100}\OperatorTok{,}
  \DataTypeTok{value}\OperatorTok{:} \DecValTok{80}\OperatorTok{,}
  \DataTypeTok{step}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}
  \DataTypeTok{onChange}\OperatorTok{:}\NormalTok{ daterangeVectors}\OperatorTok{,}
  \DataTypeTok{style}\OperatorTok{:}\NormalTok{ \{ }\DataTypeTok{width}\OperatorTok{:} \StringTok{"70\%"}\NormalTok{ \}}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}

\CommentTok{// Create a label for the slider}
\KeywordTok{var}\NormalTok{ scaleLabel }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\FunctionTok{Label}\NormalTok{(}\StringTok{"Ship Size: "}\NormalTok{)}\OperatorTok{;}

\CommentTok{// Create a panel that contains the slider and its label}
\KeywordTok{var}\NormalTok{ scalePanel }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\FunctionTok{Panel}\NormalTok{(\{}
  \DataTypeTok{widgets}\OperatorTok{:}\NormalTok{ [scaleLabel}\OperatorTok{,}\NormalTok{ scaleSlider]}\OperatorTok{,}
  \DataTypeTok{style}\OperatorTok{:}\NormalTok{ \{ }\DataTypeTok{stretch}\OperatorTok{:} \StringTok{"horizontal"}\NormalTok{ \}}\OperatorTok{,}
  \DataTypeTok{layout}\OperatorTok{:}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Panel}\OperatorTok{.}\AttributeTok{Layout}\OperatorTok{.}\FunctionTok{Flow}\NormalTok{(}\StringTok{"horizontal"}\NormalTok{)}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The last widget we're going to define is the date slider. This widget
will trigger the \texttt{daterangeVectors} function, which will filter
the Sentinel-1 dataset to only include images from the selected year,
and then run the detection process on the filtered dataset.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Specify the start and end dates for the date slider}
\KeywordTok{var}\NormalTok{ start }\OperatorTok{=} \StringTok{"2014{-}01{-}01"}\OperatorTok{;}
\KeywordTok{var}\NormalTok{ now }\OperatorTok{=} \BuiltInTok{Date}\OperatorTok{.}\FunctionTok{now}\NormalTok{()}\OperatorTok{;}

\CommentTok{// Create a date slider that allows the user to select a year}
\KeywordTok{var}\NormalTok{ dateSlider }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\FunctionTok{DateSlider}\NormalTok{(\{}
  \DataTypeTok{value}\OperatorTok{:} \StringTok{"2021{-}03{-}01"}\OperatorTok{,}
  \DataTypeTok{start}\OperatorTok{:}\NormalTok{ start}\OperatorTok{,}
  \DataTypeTok{end}\OperatorTok{:}\NormalTok{ now}\OperatorTok{,}
  \DataTypeTok{period}\OperatorTok{:} \DecValTok{365}\OperatorTok{,}
  \DataTypeTok{onChange}\OperatorTok{:}\NormalTok{ daterangeVectors}\OperatorTok{,}
  \DataTypeTok{style}\OperatorTok{:}\NormalTok{ \{ }\DataTypeTok{width}\OperatorTok{:} \StringTok{"95\%"}\NormalTok{ \}}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

\hypertarget{the-control-panel}{%
\subsection*{The Control Panel}\label{the-control-panel}}
\addcontentsline{toc}{subsection}{The Control Panel}

Now we're going to assemble all of the widgets we've just defined into
one panel, alongsie some explanatory text. I'm adding a blank label to
the panel as a placeholder for the chart, since it will be re-added to
the panel every time the user changed the date on the date slider, the
AOI, or the scale.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{var}\NormalTok{ controlPanel }\OperatorTok{=}\NormalTok{ ui}\OperatorTok{.}\FunctionTok{Panel}\NormalTok{(\{}
  \DataTypeTok{widgets}\OperatorTok{:}\NormalTok{ [}
\NormalTok{    ui}\OperatorTok{.}\FunctionTok{Label}\NormalTok{(}\StringTok{"SAR Ship Detection"}\OperatorTok{,}\NormalTok{ \{}
      \DataTypeTok{fontWeight}\OperatorTok{:} \StringTok{"bold"}\OperatorTok{,}
      \DataTypeTok{fontSize}\OperatorTok{:} \StringTok{"20px"}\OperatorTok{,}
\NormalTok{    \})}\OperatorTok{,}
\NormalTok{    ui}\OperatorTok{.}\FunctionTok{Label}\NormalTok{(}
      \StringTok{"This tool identifies ships using Synthetic Aperture Radar imagery. Use the date slider below to analyze a given year. Click on the graph to show ships on a given day."}\OperatorTok{,}
\NormalTok{      \{ }\DataTypeTok{whiteSpace}\OperatorTok{:} \StringTok{"wrap"}\NormalTok{ \}}
\NormalTok{    )}\OperatorTok{,}
\NormalTok{    dateSlider}\OperatorTok{,}
\NormalTok{    ui}\OperatorTok{.}\FunctionTok{Label}\NormalTok{()}\OperatorTok{,}
\NormalTok{    scalePanel}\OperatorTok{,}
\NormalTok{    ui}\OperatorTok{.}\FunctionTok{Label}\NormalTok{(}
      \StringTok{"Click the button below and draw a rectangle on the map to count ships in a custom area."}
\NormalTok{    )}\OperatorTok{,}
\NormalTok{    drawButton}
\NormalTok{  ]}\OperatorTok{,}
  \DataTypeTok{style}\OperatorTok{:}\NormalTok{ \{}\DataTypeTok{maxWidth}\OperatorTok{:} \StringTok{"400px"}\NormalTok{\}}\OperatorTok{,}
  \DataTypeTok{layout}\OperatorTok{:}\NormalTok{ ui}\OperatorTok{.}\AttributeTok{Panel}\OperatorTok{.}\AttributeTok{Layout}\OperatorTok{.}\FunctionTok{flow}\NormalTok{(}\StringTok{"vertical"}\OperatorTok{,} \KeywordTok{true}\NormalTok{)}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Once the control panel has been defined, we can add it to

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Add the control panel to the map}
\NormalTok{ui}\OperatorTok{.}\AttributeTok{root}\OperatorTok{.}\FunctionTok{insert}\NormalTok{(}\DecValTok{0}\OperatorTok{,}\NormalTok{controlPanel)}\OperatorTok{;}

\CommentTok{// Trigger the daterangeVectors function when the user draws a polygon}
\NormalTok{drawingTools}\OperatorTok{.}\FunctionTok{onDraw}\NormalTok{(ui}\OperatorTok{.}\AttributeTok{util}\OperatorTok{.}\FunctionTok{debounce}\NormalTok{(daterangeVectors}\OperatorTok{,} \DecValTok{500}\NormalTok{))}\OperatorTok{;}

\CommentTok{// Run the daterangeVectors function to initialize the map}
\FunctionTok{daterangeVectors}\NormalTok{()}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

And there we have it. A fully functional, all weather, daytime/nighttime
ship detection tool that doesn't rely on AIS data. Let's play around
with it.

\hypertarget{taking-it-for-a-spin}{%
\section*{Taking it for a spin}\label{taking-it-for-a-spin}}
\addcontentsline{toc}{section}{Taking it for a spin}

\markright{Taking it for a spin}

\hypertarget{north-korea}{%
\subsection*{North Korea}\label{north-korea}}
\addcontentsline{toc}{subsection}{North Korea}

Information on North Korea's economy is pretty hard to come by. Ship
traffic in and out of the country's largest port, Nampo, is probably a
pretty good indicator of the country's economic activity. But we can't
just head on down to Marine Tracker or other services that use AIS data
to track ship movements. According to the
\href{https://home.treasury.gov/system/files/126/dprk_vessel_advisory_02232018.pdf}{U.S.
Treasury}, ``North Korean-flagged merchant vessels have been known to
intentionally disable their AIS transponders to mask their movements.
This tactic, whether employed by North Korean-flagged vessels or other
vessels involved in trade with North Korea, could conceal the origin or
destination of cargo destined for, or originating in, North Korea.''
They should know-- they're the ones imposing the sanctions that make it
illegal to trade with North Korea.

A New York Times
\href{https://www.nytimes.com/2019/07/16/world/asia/north-korea-luxury-goods-sanctions.html}{investigation}
tracked the maritime voyage of luxury Mercedes cars from Germany to
North Korea via the Netherlands, China, Japan, South Korea, and Russia.
AIS transponders were turned off at several points throughout this
journey, and the investigation had to rely on satellite imagery to fill
in the gaps. Though they used high resolution optical imagery to follow
individual ships,

\includegraphics{images/ships_north_korea.jpg}

\hypertarget{ukraine-1}{%
\section*{Ukraine}\label{ukraine-1}}
\addcontentsline{toc}{section}{Ukraine}

\markright{Ukraine}

The port of Odessa is Ukraine's largest port, and Following its invasion
of Ukraine in February 2022, Russia instituted a naval blockade against
Ukrainian ports. The impact of this blockade is clearly visible using
the tool we've just built:

\includegraphics{images/ships_ukraine.jpg}

The daily number of ships detected in the port of Odessa dropped from
40-50 to 0-5 following the invasion, and remained near zero until the
blockade was lifted in September 2022.

\hypertarget{blast-damage-assessment}{%
\chapter*{Blast Damage Assessment}\label{blast-damage-assessment}}
\addcontentsline{toc}{chapter}{Blast Damage Assessment}

\markboth{Blast Damage Assessment}{Blast Damage Assessment}

On August 4th, 2020, a warehouse full of fertilizer in the port of
Beirut exploded. The blast killed over 200 people, injured thousands,
and caused widespread damage to the city:

Assessing blast damage is a common task for open source investigators,
and satellite imagery analysis is one of the best tools we have at our
disposal to analyze this sort of phenomenon.
\href{https://earthobservatory.nasa.gov/images/147098/scientists-map-beirut-blast-damage}{NASA}
used Sentinel-1 imagery in the aftermath of the explosion to generate an
estimated damage map. They explain that Sentinel-1 Synthetic Aperture
Radar (SAR) imagery is good for this sort of task:

``SAR instruments send pulses of microwaves toward Earth's surface and
listen for the reflections of those waves. The radar waves can penetrate
cloud cover, vegetation, and the dark of night to detect changes that
might not show up in visible light imagery. When Earth's crust moves due
to an earthquake, when dry land is suddenly covered by flood water, or
when buildings have been damaged or toppled, the amplitude and phase of
radar wave reflections changes in those areas and indicates to the
satellite that something on the ground has changed.''

The NASA team produced this estimate the day after the explosion, which
is very impressive. However, due to the quick turnaround, were pretty
light on the description of their methodology (they didnt provide any
code, or even explicity say how exactly they generated the estimate),
making their analysis hard to replicate. They also failed to validate
their results, which is a critical step in any analysis.

In this case study we'll be developing our own change detection
algorithm from scratch, applying to Sentinel-1 imagery of Beirut before
and after the blast, and validating our results using building
footprints and U.N. damage estimates as the ground truth. Below is the
final result of the analysis, which shows building footprints colored
according to the predicted level of damage they sustained from the
blast:

\begin{figure*}

\end{figure*}

\hypertarget{change-detection-1}{%
\section*{Change Detection}\label{change-detection-1}}
\addcontentsline{toc}{section}{Change Detection}

\markright{Change Detection}

There are many ways to detect change between two images. The simplest
way would be to take an image taken before an event, and subtract it
from an image taken after. This is a good way to get a general sense of
where change has occurred, but if you only use two images (one before an
event and another after), it would be difficult to differentiate between
areas that have changed as a result of the event in question, and areas
that have changed for other reasons. Things in Beirut (and cities in
general) are constantly changing: construction, cars/planes/ships
moving, vegetation growing, etc. So we wouldn't know if the change we're
seeing is a result of the explosion or whether that area is generally
prone to change. We can overcome this by comparing a bunch of pre-event
images to a bunch of post-event images. This way we can see if the
change we're seeing is consistent across all of the images. If it is,
then we can be fairly confident that the change is a result of the event
in question. The mean is simply the sum of all the values (\(x_i\)) in a
set divided by the number of values (\(n\)):

\[\large \overline{x} = \frac{1}{n} \sum_{i=1}^n x_i\]

But if we just take the average pixel value before and subtract the
average pixel value after, we're not accounting for the
\emph{variability} of that pixel's values. For example, if we have a
pixel that has had an average value of 1 for the month before the event,
and a value of 2 in the month after the event, the difference is 1. If
that pixel's value is extremely consistent (it never varies by more than
0.1), such a change would be very significant. But if that pixel's value
is very variable (it varies by 2 or even 3 on a regular basis), then the
change is not significant. So we need to account for the variability of
the pixel's values using the standard deviation. It is calculated as the
square root of the variance, which is the average of the squared
differences from the mean:

\[\large s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}\]

With just the mean and the standard deviations of two sets of numbers,
we can use a statistical test to determine whether the change in means
is significant. The simplest way to do this is to use a pixelwise
t-test, which is basically just a signal-to-noise ratio: it calculates
the difference between two sample means (signal), and divides it by the
standard deviations of both samples (noise). In this case, the two
samples are the pre- and post-event images. The t-test is applied to
each pixel in the image, allowing us to determine whether the change is
statistically significant. Given two groups, \(x_1\) before the event,
and \(x_2\) after the event, the \(t\) statistic is calculated as:

\[ \Large t = {\frac{\overline{x_1}-\overline{x_2}} {\sqrt{\frac{s^2_1}{n_1} + \frac{s^2_2}{n_2}}}} \]

Where:

\begin{itemize}
\tightlist
\item
  \(\overline{x}\): Sample Mean
\item
  \(s^2\): Sample Standard Deviation
\item
  \(n\): Number of observations
\end{itemize}

This procedure gives us a number called a t-value, which is a measure of
how many standard deviations the difference between the two means is.
We're not going to get into the details here, but a rule of thumb is
that if the t-value is greater than 2, then the difference between the
two means is significant. If the t-value is less than 2, then the
difference is not significant. We're going to calculate the t-value for
each pixel in the image to determine whether that pixel has changed
significantly following the event in question. You don't need to know
the details of the t-test to understand the results (but hopefully
you've got an intuition for what it's doing). If you're interested in
learning more about statistical tests of this sort, I teach a course on
Data Science at the University College London, and have made all of the
lectures and courseware open-source. The T-test lecture is here.

\hypertarget{implementing-a-t-test-in-earth-engine}{%
\section*{Implementing a t-test in Earth
Engine}\label{implementing-a-t-test-in-earth-engine}}
\addcontentsline{toc}{section}{Implementing a t-test in Earth Engine}

\markright{Implementing a t-test in Earth Engine}

Now lets go about implementing this in Earth Engine. We'll start by
centering the map on the port of Beirut, and setting the map to
satellite view, and defining an area of interest (AOI) as a 3km buffer
around the port:

\begin{Shaded}
\begin{Highlighting}[]
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setCenter}\NormalTok{(}\FloatTok{35.51898}\OperatorTok{,} \FloatTok{33.90153}\OperatorTok{,} \DecValTok{15}\NormalTok{)}\OperatorTok{;}

\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{setOptions}\NormalTok{(}\StringTok{"satellite"}\NormalTok{)}\OperatorTok{;}

\KeywordTok{var}\NormalTok{ aoi }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Geometry}\OperatorTok{.}\FunctionTok{Point}\NormalTok{(}\FloatTok{35.51898}\OperatorTok{,} \FloatTok{33.90153}\NormalTok{)}\OperatorTok{.}\FunctionTok{buffer}\NormalTok{(}\DecValTok{3000}\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

Next, let's define a function in earth engine that will perform the
T-Test. The block of code below defines a function to implement a t-test
for every pixel in a set of images. The function will be called `ttest',
and takes four arguments:

\begin{itemize}
\tightlist
\item
  s1: the image collection
\item
  shock: the date of the event
\item
  pre\_interval: the number of months before the event
\item
  post\_interval: the number of months after the event
\end{itemize}

The function will return a t-value image, which we can use to determine
whether a pixel has changed significantly following the event in
question.

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{function} \FunctionTok{ttest}\NormalTok{(s1}\OperatorTok{,}\NormalTok{ shock}\OperatorTok{,}\NormalTok{ pre\_interval}\OperatorTok{,}\NormalTok{ post\_interval) \{}
  
  \CommentTok{// Convert the shock date to a date object}
  \KeywordTok{var}\NormalTok{ shock }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Date}\NormalTok{(shock)}\OperatorTok{;}
  \CommentTok{// Filter the image collection to the pre{-}event period}
  \KeywordTok{var}\NormalTok{ pre }\OperatorTok{=}\NormalTok{ s1}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(}
\NormalTok{    shock}\OperatorTok{.}\FunctionTok{advance}\NormalTok{(ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(pre\_interval)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}\OperatorTok{{-}}\DecValTok{1}\NormalTok{)}\OperatorTok{,} \StringTok{"month"}\NormalTok{)}\OperatorTok{,}
\NormalTok{    shock}
\NormalTok{  )}\OperatorTok{;}
  \CommentTok{// Filter the image collection to the post{-}event period}
  \KeywordTok{var}\NormalTok{ post }\OperatorTok{=}\NormalTok{ s1}\OperatorTok{.}\FunctionTok{filterDate}\NormalTok{(shock}\OperatorTok{,}\NormalTok{ shock}\OperatorTok{.}\FunctionTok{advance}\NormalTok{(post\_interval}\OperatorTok{,} \StringTok{"month"}\NormalTok{))}\OperatorTok{;}
  
  \CommentTok{// Calculate the mean, standard deviation, and number of images for the pre{-}event period}
  \KeywordTok{var}\NormalTok{ pre\_mean }\OperatorTok{=}\NormalTok{ pre}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{;}
  \KeywordTok{var}\NormalTok{ pre\_sd }\OperatorTok{=}\NormalTok{ pre}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{stdDev}\NormalTok{())}\OperatorTok{;}
  \KeywordTok{var}\NormalTok{ pre\_n }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(pre}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(aoi)}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
  
  \CommentTok{// Calculate the mean, standard deviation, and number of images for the pre{-}event period}
  \KeywordTok{var}\NormalTok{ post\_mean }\OperatorTok{=}\NormalTok{ post}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{;}
  \KeywordTok{var}\NormalTok{ post\_sd }\OperatorTok{=}\NormalTok{ post}\OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{stdDev}\NormalTok{())}\OperatorTok{;}
  \KeywordTok{var}\NormalTok{ post\_n }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(post}\OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(aoi)}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
  
  \CommentTok{// Calculate the pooled standard deviation}
  \KeywordTok{var}\NormalTok{ pooled\_sd }\OperatorTok{=}\NormalTok{ pre\_sd}
    \OperatorTok{.}\FunctionTok{multiply}\NormalTok{(pre\_sd)}
    \OperatorTok{.}\FunctionTok{multiply}\NormalTok{(pre\_n}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(}\DecValTok{1}\NormalTok{))}
    \OperatorTok{.}\FunctionTok{add}\NormalTok{(post\_sd}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(post\_sd)}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(post\_n}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(}\DecValTok{1}\NormalTok{)))}
    \OperatorTok{.}\FunctionTok{divide}\NormalTok{(pre\_n}\OperatorTok{.}\FunctionTok{add}\NormalTok{(post\_n)}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(}\DecValTok{2}\NormalTok{))}
    \OperatorTok{.}\FunctionTok{sqrt}\NormalTok{()}\OperatorTok{;}

    \CommentTok{// Calculate the denominator of the t{-}test}
  \KeywordTok{var}\NormalTok{ denom }\OperatorTok{=}\NormalTok{ pooled\_sd}\OperatorTok{.}\FunctionTok{multiply}\NormalTok{(}
\NormalTok{    ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(pre\_n)}\OperatorTok{.}\FunctionTok{add}\NormalTok{(ee}\OperatorTok{.}\FunctionTok{Number}\NormalTok{(}\DecValTok{1}\NormalTok{)}\OperatorTok{.}\FunctionTok{divide}\NormalTok{(post\_n))}\OperatorTok{.}\FunctionTok{sqrt}\NormalTok{()}
\NormalTok{  )}\OperatorTok{;}

    \CommentTok{// Calculate the Degrees of Freedom, which is the number of observations minus 2}
  \KeywordTok{var}\NormalTok{ df }\OperatorTok{=}\NormalTok{ pre\_n}\OperatorTok{.}\FunctionTok{add}\NormalTok{(post\_n)}\OperatorTok{.}\FunctionTok{subtract}\NormalTok{(}\DecValTok{2}\NormalTok{)}\OperatorTok{;}

  \FunctionTok{print}\NormalTok{(}\StringTok{"Number of Images: "}\OperatorTok{,}\NormalTok{ df)}\OperatorTok{;}

    \CommentTok{// Calculate the t{-}test using the:}
        \CommentTok{// mean of the pre{-}event period, }
        \CommentTok{// the mean of the post{-}event period, }
        \CommentTok{// and the pooled standard deviation}
  \KeywordTok{var}\NormalTok{ change }\OperatorTok{=}\NormalTok{ post\_mean}
    \OperatorTok{.}\FunctionTok{abs}\NormalTok{()}
    \OperatorTok{.}\FunctionTok{subtract}\NormalTok{(pre\_mean}\OperatorTok{.}\FunctionTok{abs}\NormalTok{())}
    \OperatorTok{.}\FunctionTok{divide}\NormalTok{(denom)}
    \OperatorTok{.}\FunctionTok{abs}\NormalTok{()}
    \OperatorTok{.}\FunctionTok{subtract}\NormalTok{(}\DecValTok{2}\NormalTok{)}\OperatorTok{;}

    \CommentTok{// return the t{-}values for each pixel}
  \ControlFlowTok{return}\NormalTok{ change}\OperatorTok{;}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

An important detail in the code above is that we've actually tweaked the
t-test slightly, in two ways.

First, the algorithm above returns tha \emph{absolute} value of t
(i.e.~the absolute value of the difference between the two means). This
is because we're interested in whether the pixel has changed at all, not
whether it's changed in a particular direction. Second, we've subtracted
2 from the t-value.

The t-value is a measure of how many standard deviations the difference
between the two means is. Generally speaking, if the t-value is greater
than 2, then the difference between the two means is considered
statistically significant. 2 is a fairly abitrary cutoff, but it's the
most commonly used one since it corresponds to the 95\% confidence
interval (i.e., theres less than a 5\% chance of observing a difference
that big due to random chance). Now we've got a function that can take
an image collection and return a t-value image, where a value greater
than 0 corresponds to a statistically significant change between the
pre-event and post-event periods.

\hypertarget{filtering-the-sentinel-1-imagery}{%
\section*{Filtering the Sentinel-1
Imagery}\label{filtering-the-sentinel-1-imagery}}
\addcontentsline{toc}{section}{Filtering the Sentinel-1 Imagery}

\markright{Filtering the Sentinel-1 Imagery}

We can't just blindly apply this algorithm to the entire image
collection, because the image collection contains images from both
ascending and descending orbits. We need to filter the image collection
to the ascending and descending orbits, and then calculate the t-value
for each orbit separately: this is because the satellite is viewing the
scene from a completely different angle when it's in ascending and
descending orbits, which will generate a lot of noise in our data. In
fact, even when the satellite is either ascending or descending, we can
have multiple images of the same place taken from slightly different
orbital tracks because these overlap (see \href{/ch1\#orbits}{this
visualization of orbits}). We need to filter the image collection to the
relative orbit number that is most common within the image collection.
For that, we define a new function called `filter\_s1', which takes a
single argument: the path (either `ASCENDING' or `DESCENDING').

\begin{Shaded}
\begin{Highlighting}[]
\KeywordTok{function} \FunctionTok{filter\_s1}\NormalTok{(path) \{}
  
  \CommentTok{// Filter the image collection to the ascending or descending orbit}
  \KeywordTok{var}\NormalTok{ s1 }\OperatorTok{=}\NormalTok{ ee}
    \OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{(}\StringTok{"COPERNICUS/S1\_GRD"}\NormalTok{)}
    \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{listContains}\NormalTok{(}\StringTok{"transmitterReceiverPolarisation"}\OperatorTok{,} \StringTok{"VH"}\NormalTok{))}
    \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{"instrumentMode"}\OperatorTok{,} \StringTok{"IW"}\NormalTok{))}
    \OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{"orbitProperties\_pass"}\OperatorTok{,}\NormalTok{ path))}
    \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(aoi)}
    \OperatorTok{.}\FunctionTok{select}\NormalTok{(}\StringTok{"VH"}\NormalTok{)}\OperatorTok{;}

    \CommentTok{// Find the most common relative orbit number}
  \KeywordTok{var}\NormalTok{ orbit }\OperatorTok{=}\NormalTok{ s1}
    \OperatorTok{.}\FunctionTok{aggregate\_array}\NormalTok{(}\StringTok{"relativeOrbitNumber\_start"}\NormalTok{)}
    \OperatorTok{.}\FunctionTok{reduce}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mode}\NormalTok{())}\OperatorTok{;}

    \CommentTok{// Filter the image collection to the most common relative orbit number}
  \KeywordTok{var}\NormalTok{ s1 }\OperatorTok{=}\NormalTok{ s1}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{eq}\NormalTok{(}\StringTok{"relativeOrbitNumber\_start"}\OperatorTok{,}\NormalTok{ orbit))}\OperatorTok{;}

    \CommentTok{// Calculate the t{-}test for the filtered image collection using the function we defined earlier}
  \KeywordTok{var}\NormalTok{ change }\OperatorTok{=} \FunctionTok{ttest}\NormalTok{(s1}\OperatorTok{,} \StringTok{"2020{-}08{-}04"}\OperatorTok{,} \DecValTok{12}\OperatorTok{,} \DecValTok{2}\NormalTok{)}\OperatorTok{;}

    \CommentTok{// Return the t{-}values for each pixel}
  \ControlFlowTok{return}\NormalTok{ change}\OperatorTok{;}
\NormalTok{\}}
\end{Highlighting}
\end{Shaded}

You'll notice that we've called the ttest function we defined earlier
with four arguments:

\begin{itemize}
\tightlist
\item
  s1: the Sentinel-1 image collection filtered to the ascending or
  descending orbit
\item
  2020-08-04: the date of the explosion
\item
  12: the number of months before the explosion to use for the pre-event
  period; I'm choosing to include the full year prior to the explosion
  to get a good baseline
\item
  2: the number of months after the explosion to use for the post-event
  period; I'm including 2 months after the explosion. Much less than
  that, and we risk missing the effects of the explosion. Much more than
  that, and we risk including the effects other changes that happened
  after the explosion, including the reconstruction effort.
\end{itemize}

Now we want to apply this function to the image collection twice (once
for each orbit) and then combine the two images into a single image.
After that, we can clip it to the area of interest and display it on the
map:

\begin{Shaded}
\begin{Highlighting}[]

\CommentTok{// Call the filter\_s1 function twice, once for each orbit, and then combine the two images into a single image}
\KeywordTok{var}\NormalTok{ composite }\OperatorTok{=}\NormalTok{ ee}
  \OperatorTok{.}\FunctionTok{ImageCollection}\NormalTok{([}\FunctionTok{filter\_s1}\NormalTok{(}\StringTok{"ASCENDING"}\NormalTok{)}\OperatorTok{,} \FunctionTok{filter\_s1}\NormalTok{(}\StringTok{"DESCENDING"}\NormalTok{)])}
  \OperatorTok{.}\FunctionTok{mean}\NormalTok{()}
  \OperatorTok{.}\FunctionTok{clip}\NormalTok{(aoi)}\OperatorTok{;}

\CommentTok{// Define a color palette}
\KeywordTok{var}\NormalTok{ palette }\OperatorTok{=}\NormalTok{ [}\StringTok{"440154"}\OperatorTok{,} \StringTok{"3b528b"}\OperatorTok{,} \StringTok{"21918c"}\OperatorTok{,} \StringTok{"5ec962"}\OperatorTok{,} \StringTok{"fde725"}\NormalTok{]}\OperatorTok{;}

\CommentTok{// Add the composite to the map}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(}
\NormalTok{  composite}\OperatorTok{,}
\NormalTok{  \{ }\DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,} \DataTypeTok{max}\OperatorTok{:} \DecValTok{4}\OperatorTok{,} \DataTypeTok{opacity}\OperatorTok{:} \FloatTok{0.8}\OperatorTok{,} \DataTypeTok{palette}\OperatorTok{:}\NormalTok{ palette \}}\OperatorTok{,}
  \StringTok{"change"}
\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The visualization parameters correspond the statitical significance of
the change in pixel values. Using the Viridis color palette which ranges
from purple to yellow, dark purple pixels indicate no significant
change, and yellow pixels indicate a significant change with with 95\%
confidence. The brighter the yellow of a pixel, the more significant the
change.

\begin{figure}

{\centering \includegraphics{././images/beirut/beirut_change_2020.jpg}

}

\caption{Pixelwise T-Test, 2020}

\end{figure}

This seems to be working quite well; but remember, ports are generally
prone to change. The t-test is accounting for this by calculating each
pixel's variance over the entire time period, but it's still possible
that the change we're seeing is due to the port rather than the
explosion. To test this, we can run the same algorithm on the same area,
using the same date cutoff (August 4th), but in a different year; i've
chosen 2018. This is what's known as a placebo test: if it's still
showing loads of statistically significant change around the cutoff, our
algorithm is probably picking up on port activity rather than the
explosion.

\begin{figure}

{\centering \includegraphics{././images/beirut/beirut_change_2018.jpg}

}

\caption{Pixelwise T-Test, 2018}

\end{figure}

Compared to the 2020 image, there's a lot less yellow (significant
change). That being said there are a few yellow areas. This could be due
to a number of reasons: ships coming and going, cranes moving, and
containers being loaded and unloaded would all register in the change
detection algorithm. There are also a number of yellow specks throughout
the city, which is also to be expected since cities are also generally
in a state of flux. Construction, demolition, and even the growth of
vegetation can all be detected by the algorithm.

However, the scale and quantity of the change is nowhere near what it
was for the 2020 image. This is a good sign that the algorithm detecting
change resulting from the explosion.

\hypertarget{validation-1}{%
\section*{Validation}\label{validation-1}}
\addcontentsline{toc}{section}{Validation}

\markright{Validation}

Great. We've developed our very own change detection algorithm in earth
engine, applied it to the Beirut explosion, and it seems to be working
using a basic placebo test. But how do we know that it's correctly
predicting the \emph{extent} of the damage, and not wildly
over/underestimating?

Given that this was a few years ago, we have the benefit of hindsight.
In particular, the United Nations and the Municipality of Beirut have
\href{https://unhabitat.org/sites/default/files/2020/10/municipality_of_beirut_-_beirut_explosion_rapid_assessment_report.pdf}{published
a report} on the damage caused by the explosion. This report includes
estimates of the number of buildings damaged or destroyed by the
explosion, as well as the number of people displaced. The report states
that approximately 10,000 buildings were damaged within a 3km radius of
the port. If our algorithm suggests that only 1,000 buildings were
damaged, it's undershooting. If it suggests that 100,000 buildings were
damaged, it's overshooting.

Using building footprint data and the t-test image we just generated, we
can generate an estimate of the number of damaged buildings according to
our model. First, we want to generate a thresholded image, where pixels
with a value greater than 0 are set to 1, and all other pixels are set
to 0. We can then use this mask to reduce the building footprints to a
single value for each building, where the value is the mean of the
t-test image within the footprint. If the mean value is greater than 0,
the building is damaged. If it's less than 0, the building is not
damaged.

\begin{Shaded}
\begin{Highlighting}[]
\CommentTok{// Create a mask of the t{-}test image, where pixels with a value greater than 0 are set to 1, and all other pixels are set to 0}
\KeywordTok{var}\NormalTok{ threshold }\OperatorTok{=}\NormalTok{ composite}\OperatorTok{.}\FunctionTok{updateMask}\NormalTok{(composite}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\DecValTok{0}\NormalTok{))}\OperatorTok{;}

\CommentTok{// Load the building footprints}
\KeywordTok{var}\NormalTok{ buildings }\OperatorTok{=}\NormalTok{ ee}
  \OperatorTok{.}\FunctionTok{FeatureCollection}\NormalTok{(}\StringTok{"projects/sat{-}io/open{-}datasets/MSBuildings/Lebanon"}\NormalTok{)}
  \OperatorTok{.}\FunctionTok{filterBounds}\NormalTok{(aoi)}\OperatorTok{;}

\CommentTok{// Calculate the mean value of the t{-}test image within each building footprint}
\KeywordTok{var}\NormalTok{ damaged\_buildings }\OperatorTok{=}\NormalTok{ threshold}\OperatorTok{.}\FunctionTok{reduceRegions}\NormalTok{(\{}
  \DataTypeTok{collection}\OperatorTok{:}\NormalTok{ buildings}\OperatorTok{,}
  \DataTypeTok{reducer}\OperatorTok{:}\NormalTok{ ee}\OperatorTok{.}\AttributeTok{Reducer}\OperatorTok{.}\FunctionTok{mean}\NormalTok{()}\OperatorTok{,}
  \DataTypeTok{scale}\OperatorTok{:} \DecValTok{1}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}

\CommentTok{// Print the number of buildings with a mean value greater than 0}
\CommentTok{// i.e., those displaying statistically  significant change}
\FunctionTok{print}\NormalTok{(damaged\_buildings}\OperatorTok{.}\FunctionTok{filter}\NormalTok{(ee}\OperatorTok{.}\AttributeTok{Filter}\OperatorTok{.}\FunctionTok{gt}\NormalTok{(}\StringTok{"mean"}\OperatorTok{,} \DecValTok{0}\NormalTok{))}\OperatorTok{.}\FunctionTok{size}\NormalTok{())}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result is 9,256, which is pretty damn close to 10,000. We can also
visualize the building footprints on the map, colored according the mean
value of the t-test image within the footprint, where:

\begin{itemize}
\tightlist
\item
  Blue = no damage
\item
  Green = low damage
\item
  Yellow/Orange = medium damage
\item
  Red = high levels of damage
\end{itemize}

\begin{Shaded}
\begin{Highlighting}[]

\CommentTok{// Create an empty image}
\KeywordTok{var}\NormalTok{ empty }\OperatorTok{=}\NormalTok{ ee}\OperatorTok{.}\FunctionTok{Image}\NormalTok{()}\OperatorTok{.}\FunctionTok{byte}\NormalTok{()}\OperatorTok{;}

\CommentTok{// Paint the building footprints onto the empty image}
\KeywordTok{var}\NormalTok{ outline }\OperatorTok{=}\NormalTok{ empty}\OperatorTok{.}\FunctionTok{paint}\NormalTok{(\{}
  \DataTypeTok{featureCollection}\OperatorTok{:}\NormalTok{ damaged\_buildings}\OperatorTok{,}
  \DataTypeTok{color}\OperatorTok{:} \StringTok{"mean"}\OperatorTok{,}
  \DataTypeTok{width}\OperatorTok{:} \DecValTok{5}\OperatorTok{,}
\NormalTok{\})}\OperatorTok{;}

\CommentTok{// Define a color palette}
\KeywordTok{var}\NormalTok{ building\_palette }\OperatorTok{=}\NormalTok{ [}
  \StringTok{"0034f5"}\OperatorTok{,}
  \StringTok{"1e7d83"}\OperatorTok{,}
  \StringTok{"4da910"}\OperatorTok{,}
  \StringTok{"b3c120"}\OperatorTok{,}
  \StringTok{"fcc228"}\OperatorTok{,}
  \StringTok{"ff8410"}\OperatorTok{,}
  \StringTok{"fd3000"}\OperatorTok{,}
\NormalTok{]}\OperatorTok{;}

\CommentTok{// Add the image to the map}
\BuiltInTok{Map}\OperatorTok{.}\FunctionTok{addLayer}\NormalTok{(}
\NormalTok{  outline}\OperatorTok{,}
\NormalTok{  \{ }\DataTypeTok{palette}\OperatorTok{:}\NormalTok{ building\_palette}\OperatorTok{,} \DataTypeTok{min}\OperatorTok{:} \DecValTok{0}\OperatorTok{,} \DataTypeTok{max}\OperatorTok{:} \DecValTok{2}\NormalTok{ \}}\OperatorTok{,}
  \StringTok{"Damaged Buildings"}
\NormalTok{)}\OperatorTok{;}
\end{Highlighting}
\end{Shaded}

The result naturally resembles the underlying t-test image, with high
levels of damage concetrated around the port, and progressively
decreasing damage with distance:

\begin{figure}

{\centering \includegraphics{././images/beirut/beirut_footprints.jpg}

}

\caption{Building Footprints colored according to estimated blast
damage}

\end{figure}

To get a better sense of how these predicitons correspond to actual
damage, we can zoom in and turn on the Google satellite basemap, which
has imagery taken just after the explosion; you can still see capsized
boats in the port. Zooming in to the epicentre, we can see several
warehouses that were effectively vaporized. Our change detection
algorithm picks up on a high degree of change, as indicated by the red
outlines of the building footprints:

\begin{figure}

{\centering \includegraphics{././images/beirut/beirut_footprints_port.jpg}

}

\caption{Predicted damage and optical satellite imagery in the Port of
Beirut, August 2020}

\end{figure}

This is pretty low-hanging fruit. Let's look at a different area, around
1.3km east from the epicentre with a mix of warehouses and residential
buildings:

\begin{figure}

{\centering \includegraphics{././images/beirut/beirut_footprints_zoomed.jpg}

}

\caption{Area east of the port: 35.533194, 33.9024}

\end{figure}

Here, there's greater variation in the predictions. I've highlighted
three areas.

In Area A, we see a warehouse with a highily deformed roof; panels of
corrugated iron are missing, and much of the roof is warped. The
building footprint for this warehouse is red, suggesting that our
algorithm correctly predicts a significant amount of blast damage.

In Area B, we see a medium-rise building. If you look closely at the
southern edge of the building, you'll see the siding has been completely
torn off and is laying on the sidewalk. The bulding footprint is orange,
suggesting a medium amount of change. We may be underestimating a bit
here.

In Area C, there are a bunch of high rise buildings clustered together.
The building footprints are all blue, suggesting little to no damage.
This is a bit of a surprise given how damaged areas A and B are. If you
squint at the satellite image, it is indeed hard to tell if these
buildings are damaged because we're looking at them from the top down,
when much of the damage (e.g., the windows being blown out) would only
be visible from the side. Indeed, our own estimate of the number of
damaged buildings based on the algorithm we developed is about 8\% shy
of the U.N.'s estimate. This may be why.

\hypertarget{conclusion-25}{%
\section*{Conclusion}\label{conclusion-25}}
\addcontentsline{toc}{section}{Conclusion}

\markright{Conclusion}

In this practical, we created a custom change detection algorithm that
conducts a pixelwise t-test to detect change resulting from the 2020
explosion in the port of Beirut. By defining our own functions to do
most of this analysis, we can apply the same workflow quite easily to a
different context by simply moving the AOI and inputting the date of the
shock. A placebo test showed that it's not just detecting general change
in the area, but specifically change resulting from the explosion: when
we keep everythgin the same but change the year of the shock, we see
very little significant change being detected. Finally, by joining the
predicted damage map to building footprints, we come up with an estimate
of 9,256 damaged buildings, which is pretty close to the U.N.'s estimate
of 10,000. That concludes the portion of this case study that deals with
Earth Engine, but if you're interested in learning more about why we're
coming up a bit short on the damage estimate (and some different ways of
looking at the problem), read on.

\hypertarget{extension-satellite-imagery-and-its-limits}{%
\section*{Extension: Satellite Imagery and its
Limits}\label{extension-satellite-imagery-and-its-limits}}
\addcontentsline{toc}{section}{Extension: Satellite Imagery and its
Limits}

\markright{Extension: Satellite Imagery and its Limits}

Though satellite imagery analysis is undoubtedly one of the best tools
we have at our disposal to analyze this sort of phenomenon, it appears
to systematically underestimate the extent of damage in Beirut. I
outline an alternative approach using Open Street Map data to create a
3D model of Beirut and the explosion to analyze directional blast
damage. Again, we're now leaving Earth Engine and moving to Blender, so
if you're not interested in that feel free to skip ahead to the next
case study.

Below is one of the most viewed videos of the explosion:

Stunning video shows explosions just minutes ago at Beirut port
pic.twitter.com/ZjltF0VcTr

--- Borzou Daragahi 🖊🗒 ((\textbf{borzou?})) August 4, 2020

Geolocating this video was pretty simple thanks to the Greek Orthodox
church (highlighted in green below) and the road leading to it
(highlighted in blue). The red box indicates the likely location
(33.889061, 35.515909) from which the person was filming:

\includegraphics{././images/beirut/IMG_2.png}

The video shows heavy damage being sustained by areas well outside the
zones classified as damaged in the maps above (both my own and NASA's).
Indeed, substantial damage was reported several kilometers away.

Why are satellite images underestimating damage in Beirut? Satellite
images are taken from above, and are two-dimensional. Much of the damage
caused by the blast, however, was directional; the pressure wave hit the
sides of buildings, as shown in this diagram from a FEMA manual:

\includegraphics{././images/beirut/IMG_3.png}

Areas close to the explosion suffered so much damage that it could be
seen from above, but even if an apartment building had all of its
windows blown out, this would not necessarily be visible in a top-down
view. Even for radar, which does technically collect data in three
dimensions, the angle problem remains; a high resolution radar might be
able to tell you how tall an apartment complex is, but it won't give you
a clear image of all sides. Case in point: the NASA damage map was
created using Sentinel-1 SAR data. In a nutshell, damage assessment in
this case is a three-dimensional problem, and remote sensing is a
two-dimensional solution.

\hypertarget{creating-a-3d-model-of-beirut}{%
\subsection*{Creating a 3D model of
Beirut}\label{creating-a-3d-model-of-beirut}}
\addcontentsline{toc}{subsection}{Creating a 3D model of Beirut}

To create a more accurate rendering of directional blast damage, three
dimensional data are required. Data from Open Street Maps (OSM) contains
information on both the ``footprints'' (i.e., the location and shape) as
well as the height of buildings, which is enough to create a three
dimensional model of Beirut. 3D rendering was done in Blender using the
Blender-OSM add-on to import a satellite basemap, terrain raster, and
OSM data.

Geolocated videos of the blast can be used to verify and adjust the
model. Below is a side-by-side comparison of the twitter video and a 3D
rendition of OSM data:

\includegraphics{././images/beirut/IMG_4.png}

Some slight adjustments to the raw OSM data were made to achieve the
image on the right. The building footprints are generally very accurate
and comprehensive in coverage, but the building height data does
occasionally have to be adjusted manually. A simple and reliable way of
doing this is to look at the shadows cast by the building on the
satellite base map and scale accordingly. I also added a rough texture
to the buildings to help differentiate them, and added the domed roof of
the Greek Orthodox church for reference.

For good measure, a second video is geolocated following the same
procedure:

Another view of the explosions in Beirut pic.twitter.com/efT5VlpMkj

--- Borzou Daragahi 🖊🗒 ((\textbf{borzou?})) August 4, 2020

The second pier (highlighted in green) and the angle (in blue) serve as
references:

\includegraphics{././images/beirut/IMG_5.png}

The video was taken from the rooftop of a japanese restaurant called
Clap Beirut (in red above). This is confirmed by a picture of the
rooftop bar on google images, which matches the bar that can be seen at
0:02 in the twitter video. Below is a comparison of the video view and
the 3D OSM model:

\includegraphics{././images/beirut/IMG_6.png}

Though somewhat grainy, the basemap on the OSM rendering shows the same
parking lot in the foreground, the second pier, and the same two
buildings highlighted in yellow. Having created a 3D model of Beirut
using OSM data, we can now simulate how the explosion would interact
with the cityscape.

\hypertarget{using-a-viewshed-analysis-to-assess-blast-exposure}{%
\subsection*{Using a Viewshed Analysis to Assess Blast
Exposure}\label{using-a-viewshed-analysis-to-assess-blast-exposure}}
\addcontentsline{toc}{subsection}{Using a Viewshed Analysis to Assess
Blast Exposure}

As the pressure wave moved through the Beirut, some buildings bore the
full force of the explosion, while others were partially shielded by
taller structures. A viewshed analysis can be conducted to identify
surfaces that were directly exposed to the explosion by creating a
lighting object at ground zero; areas that are lit up experienced
unobstructed exposure to the blast:

\includegraphics{././images/beirut/GIF_1.gif}

Pressure waves, like sound, are capable of diffraction (beding around
small obstructions). To roughly simluate this, the lighting object is
gradually raised, allowing the light to pass ``around'' obstructions.
Warehouses on the Eastern side of the docks, as well as the first row of
apartment buildings facing the docks are immediately affected. As the
lighting object rises above the warehouse, more areas suffer direct
exposure.

Using two lighting objects-- a red one at 10 meters and a blue one at 20
meters above the warehouse at ground zero-- the intensity of the blast
in different areas is highlighted; red areas suffered direct exposure,
blue areas suffered partially obstructed exposure, and black areas were
indirectly exposed.

\includegraphics{././images/beirut/IMG_7.png}

In the immediate vicinity of the explosion the large ``L'' shaped
building (Lebanon's strategic grain reserve) is bright red, and was
barely left standing. It absorbed a large amount of the blast, shielding
areas behind it and thereby casting a long blue shadow to the West. If
one refers back to the satellite damage maps above, there appears to be
significantly less damage in the area just West of (``behind'') the
grain silo, roughly corresponding to the blue shadow above. While these
areas were still heavily damaged, they seem to have suffered less damage
than areas of equal distance to the East.

\hypertarget{accounting-for-diffraction}{%
\subsection*{Accounting for
Diffraction}\label{accounting-for-diffraction}}
\addcontentsline{toc}{subsection}{Accounting for Diffraction}

The viewshed analysis tells us which sides of a building are exposed to
the blast, but it's a pretty rough approximation of the way the pressure
wave would respond to obstacles in its path. As previously mentioned,
pressure waves behave much like sound waves or waves in water: they
bounce off of objects, move around obstructions, and gradually fade.

To get a more precise idea of the way in which the blast interacted with
the urban environment, we can model the blast as an actual wave using
the ``dynamic wave'' feature in Blender. This effectively involves
creating a two-dimensional plane, telling it to behave like water, and
simulating an object being dropped into the water. By putting an
obstruction in this plane, we can see how the wave responds to it. As an
example, the grain silo has been isolated below:

\includegraphics{././images/beirut/GIF_2.gif}

As the blast hits the side of the silo, it is reflected. Two large waves
can be seen traveling to the right: the initial blast wave, and the
reflection from the silo which rivals the initial wave in magnitude. To
the left, the wave travels around the silo but is significantly
weakened.

Broadening the focus and adding the rest of the OSM data back in, we can
observe how the pressure wave interacted with buildings on the
waterfront:

\includegraphics{././images/beirut/GIF_3.gif}

The warehouses on the docks were omitted to emphasize the interaction
between the pressure wave and the waterfront buildings; their light
metal structure and low height means they would have caused little
reflection anyway. The general pattern of the dynamic wave is consistent
with the viewshed, but adds a layer of detail. The blast is reflected
off of the silo towards the East, leading to a double hit. Though the
wave still moves around the silo to the West, the pressure is
diminished. Once the wave hits the highrises, the pattern becomes noisy
as the wave both presses forward into the mainland and is reflected back
towards the pier.

\hypertarget{modeling-the-pressure-wave}{%
\subsection*{Modeling the Pressure
Wave}\label{modeling-the-pressure-wave}}
\addcontentsline{toc}{subsection}{Modeling the Pressure Wave}

Now that we've accounted for the directionality of the blast and the
influence of buildings, we can model the pressure wave itself. An
expanding sphere centered at ground zero is used to model the
progression of the pressure wave through the city. To get a visual sense
of the blast's force, the color of the sphere will be a function of the
pressure exerted by pressure wave.

The pressure exerted by the explosion in kilopascals (kPa) at various
distances can be calculated using the DoD's Blast Effects Computer,
which allows users to input variables such as the TNT equivalent of the
ordnance, storage method, and elevation. Though there are several
estimates, the blast was likely equivalent to around 300 tons of TNT.
The direct ``incident pressure'' of the pressure wave is shown in blue.
However, pressure waves from explosions that occur on the ground are
reflected upwards, amplifying the total pressure exerted by the blast.
This ``reflected pressure'' is shown in orange:

For reference, 137 kPa results in 99\% fatalities, 68 kPa is enough to
cause structural damage to most buildings, and 20 kPa results in serious
injuries. 1-6 kPa is enough to break an average window. At 1km, the
reflected pressure of the blast (18 kPa) was still enough to seriously
injure. Precisely calculating the force exerted by an explosion is
exceptionally complicated, however, so these numbers should be treated
as rough estimates. Further analysis of the damage caused by blasts
blast can be derived from the UN's Explosion Consequences Analysis
calculator which provides distance values for different types of damage
and injuries.

Linking the values in this graph to the color of the pressure wave
sphere provides a visual representation of the blast's force as it
expands. An RGB color scale corresponds to the blast's overpressure at
three threshold values.

\includegraphics{././images/beirut/beirut.gif}

By keeping the lighting object from the viewshed analysis and placing it
within the expanding sphere of the pressure wave, we combine two key
pieces of information: the pressure exerted by the blast (the color of
the sphere), and the level of directional exposure (brightness).

Now, referring back to the two geolocated twitter videos from earlier,
we can recreate the blast in our 3D model and get some new insights.
Below is a side-by-side comparison of the first video and the 3D model:

\includegraphics{././images/beirut/GIF_5.gif}

Judging by the twitter video alone, it would be very hard to tell the
fate of the person filming or the damage caused to the building that
they were in. However, the 3D model shows that despite having an
unobstructed view of the explosion, the incident pressure of the
pressure wave had decreased significtantly by the time it reached the
viewing point. The blue-green color corresponds to roughly 15 kPa--
enough to injure and break windows, but not enough to cause structural
damage to the building.

The second twitter video was taken slightly closer to ground zero, but
the view was partially obstructed by the grain silo:

\includegraphics{././images/beirut/GIF_6.gif}

Though the pressure wave probably exerted more pressure compared to the
first angle, the partial obstruction of the grain silo likely tempered
the force of the blast.

\hypertarget{assessing-damage-to-the-skyline-tower}{%
\subsection*{Assessing Damage to the Skyline
Tower}\label{assessing-damage-to-the-skyline-tower}}
\addcontentsline{toc}{subsection}{Assessing Damage to the Skyline Tower}

As a concrete example of how this approach can be used to assess damage
(or predict it, if one had the foresight), let us consider the Skyline
Tower, pictured below following the explosion:

\includegraphics{././images/beirut/IMG_8.png}

This partial side view shows two faces of the building, labelled ``A''
and ``B'' above. Side A was nearly perpendicular to the blast, and just
600 m from ground zero. Based on the previous modeling, the pressure
wave exerted roughly 40 kPa on this side of the building. The corner
where sides A and B meet, highlighted in green, shows total destruction
of windows, removal of most siding panels, and structural damage. The
back corner, highlighted in red, shows many windows still intact,
indicating that the maximum overpressure on this side of the building
likely didn't exeed 10 kPa. In other words, standing on the front
balcony would likely have led to serious injury but standing on the back
balcony would have been relatively safe.

The animation below shows the Skyline Tower as it is hit by the pressure
wave, with sides A and B labeled:

\includegraphics{././images/beirut/GIF_7.gif}

The bright green color of the pressure wave indicates a strong
likelihood of structural damage. Side A can be seen taking a direct hit,
while side B is angled slighly away. Despite not being directly exposed
to the blast, it likely still took reflective damage from some of the
neighbouring buildings. Both the incident overpressure indicated by the
color of the sphere, as well as the relative brightness of sides A and B
both correspond closely to the observed damage taken by the Skyline
Tower.

\hypertarget{further-research}{%
\subsection*{Further Research}\label{further-research}}
\addcontentsline{toc}{subsection}{Further Research}

Though satellite imagery analysis is an indispensable tool in disaster
response, it has limitations. Urban blast damage in particular is
difficult to assess accurately because it is highly directional and much
of it cannot be seen from a bird's eye view. Using free and open source
tools, an interactive 3D model of an urban explosion can be generated,
allowing for a highly detailed investigation of directional blast
damage. This can be achieved in three steps:

First, creating a 3D model of the urban area using Blender and Open
Street Maps data. Second, conducting a viewshed analysis using lighting
objects to gauge levels of unobstructed exposure to the pressure wave.
Third, modeling the explosion using geolocated videos of the event and
ordnance calculators. For added detail, a dynamic wave analysis can be
used to more precisely model how the pressure wave interacts with
buildings.

Once properly modeled, the explosion can be viewed from any angle in the
city. The viewshed analysis can be calibrated more finely by
ground-truthing various damage levels (e.g.~broken windows) at different
locations. In the absence of an official address registry in Beirut, OSM
is already being used by the Lebanese Red Cross (donate here) to conduct
neighborhood surveys assessing blast damage. As such, this type of
damage analysis can quickly be integrated into relief efforts, adapted
to model disasters in different cities, and can even be used to simulate
the destructive potential of hypothetical explosions to promote
readiness.

Speacial thanks to my nuclear physicist brother, Sean, for making sure I
didn't commit too many crimes against Physics.



\end{document}