CCDC Visualization Notebook#

  • The Continuous Change Detection and Classification algorithm is a dense time-series temporal segmentation algorithm.

  • It takes every available observation and fits seasonal harmonic models to capture stable time periods. When a significant departure from the current model is detected, a break is placed. A new harmonic model is then fit once the change completes.

  • This notebook illustrates how to visualize and use the raw CCDC output created in the CCDCWrapper.py script.

  • It also shows how to combine two raw CCDC outputs that overlap using a linear feathering technique.

    Copyright 2024 Ian Housman

    Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

github github

Bring in CCDC data and set parameters#

#Example of how to visualize CCDC outputs using the Python visualization tools
#Adds change products and fitted harmonics from CCDC output to the viewer
#The general workflow for CCDC is to run the CCDCWrapper.py script, and then either utilize the harmonic model for a given date
#or to use the breaks for change detection. All of this is demonstrated in this example
####################################################################################################
import os,sys
sys.path.append(os.getcwd())

#Module imports
try:
    import geeViz.getImagesLib as getImagesLib
except:
    !python -m pip install geeViz
    import geeViz.getImagesLib as getImagesLib


import geeViz.changeDetectionLib as changeDetectionLib
ee = getImagesLib.ee
Map = getImagesLib.Map
Map.clearMap()
####################################################################################################
#Bring in ccdc image asset
#This is assumed to be an image of arrays that is returned from the ee.Algorithms.TemporalSegmentation.Ccdc method
ccdcImg = ee.ImageCollection('projects/lcms-292214/assets/CONUS-LCMS/Base-Learners/CCDC-Collection-1984-2022')\
          .select(['tStart','tEnd','tBreak','changeProb','red.*','nir.*','swir1.*','swir2.*','NDVI.*','NBR.*']).mosaic()


#Specify which harmonics to use when predicting the CCDC model
#CCDC exports the first 3 harmonics (1 cycle/yr, 2 cycles/yr, and 3 cycles/yr)
#If you only want to see yearly patterns, specify [1]
#If you would like a tighter fit in the predicted value, include the second or third harmonic as well [1,2,3]
whichHarmonics = [1,2,3]

#Whether to fill gaps between segments' end year and the subsequent start year to the break date
fillGaps = True

#Specify which band to use for loss and gain. 
#This is most important for the loss and gain magnitude since the year of change will be the same for all years
changeDetectionBandName = 'NDVI'

# Choose whether to show the most recent ('mostRecent') or highest magnitude ('highestMag') CCDC break
sortingMethod = 'mostRecent'
####################################################################################################
#Pull out some info about the ccdc image
startJulian = 1
endJulian = 365
startYear = 1984
endYear = 2022
print('done')

done

View and query raw CCDC outputs#

Notice the raw output is difficult to use directly. All of the information is here we need for change detection, seasonal synthetic compsites, and more

Map.clearMap()
Map.port = 1231
#Add the raw array image
Map.addLayer(ccdcImg,{},'Raw CCDC Output',True)
Map.centerObject(ccdcImg)
Map.turnOnInspector()
Map.setCenter(-86.6,35,10)
Map.view()
#Double click on map to see raw CCDC output image array values
#Notice it is difficult to interpret these values as a time series since only breaks and their respective harmonic models are stored

Adding layer: Raw CCDC Output
Starting webmap
Using default refresh token for geeView
Starting local web server at: http://localhost:1231/geeView/
HTTP server command: "c:\Users\ihousman\AppData\Local\Programs\Python\Python311\python.exe" -m http.server  1231
Done
cwd a:\GEE\gee_py_modules_package\geeViz\examples
geeView URL: http://localhost:1231/geeView/?projectID=lcms-292214&accessToken=ya29.a0AeDClZAtTHj3RFawevuI9hKa43eT9FQ1UDyrSyQOoqVGmRtWC8eM6KARg2J-Aeqp5GoK0Cpowd06ogIntFP0i9UbpGss1wnUSaBJiRL-JvXCgIQOo353wtPqf1c8cC1XKvpxY_QVzjeX85ghdkZihUSk6TZKO0MyVchO0vb0rmoaCgYKASESARESFQHGX2MiPUuspCWwMfEeWPFWdJcIBw0178

Change detection using CCDC#

CCDC defines change as a significant departure from the current seasonal model. This is a good method for detecting conversion in land cover and/or land use. It does miss more subtle and/or long-term changes that may or may not result in a conversion of land cover and/or land use.

Here, we provide a method to easily pull the change information from a raw CCDC output.

Map.clearMap()
#We will not look at more useful ways of visualizing CCDC outputs
#First, we will extract the change years and magnitude
changeObj = changeDetectionLib.ccdcChangeDetection(ccdcImg,changeDetectionBandName)
Map.addLayer(changeObj[sortingMethod]['loss']['year'],{'min':startYear,'max':endYear,'palette':changeDetectionLib.lossYearPalette},'Loss Year')
Map.addLayer(changeObj[sortingMethod]['loss']['mag'],{'min':-0.5,'max':-0.1,'palette':changeDetectionLib.lossMagPalette},'Loss Mag',False)
Map.addLayer(changeObj[sortingMethod]['gain']['year'],{'min':startYear,'max':endYear,'palette':changeDetectionLib.gainYearPalette},'Gain Year')
Map.addLayer(changeObj[sortingMethod]['gain']['mag'],{'min':0.05,'max':0.2,'palette':changeDetectionLib.gainMagPalette},'Gain Mag',False)

Map.turnOnInspector()
Map.view()
#Double click on map to see raw years of loss and gain breaks
#Notice as you zoom in the layers change since GEE is processing outputs at a given pyramid level

Adding layer: Loss Year
Adding layer: Loss Mag
Adding layer: Gain Year
Adding layer: Gain Mag
Starting webmap
Using default refresh token for geeView
Local web server at: http://localhost:1231/geeView/ already serving.
cwd a:\GEE\gee_py_modules_package\geeViz\examples
geeView URL: http://localhost:1231/geeView/?projectID=lcms-292214&accessToken=ya29.a0AeDClZC1Ko3UnQzOS0yVlomGPG-gXsLr3hSbcvOtpOPAbweZfOKjr_AOFVJKsWkc4QbyRCikS1VZ4-NA7SPSEVRtok3s1U8PqE8V2YlL-13NyhbX40bmowjokjtZPx_4gOY9_1y-6h-NIartw1yVAvSvFf_XPueFg1lVGNAuv5MaCgYKAVMSARESFQHGX2MiIi1SnuadItfSpeqOQvQw1Q0178

Create fitted time series#

This example shows a common use of CCDC - to create images at some set time interval

Specifically, this example is showing a time series for every 0.1 of a year

View the time lapse to see the green-up and brown-down over a growing season

Map.clearMap()
#Apply the CCDC harmonic model across a time series
#First get a time series of time images 
yearImages = changeDetectionLib.simpleGetTimeImageCollection(startYear,endYear,startJulian,endJulian,0.1)

#Then predict the CCDC models
fitted = changeDetectionLib.predictCCDC(ccdcImg,yearImages,fillGaps,whichHarmonics)

Map.addLayer(fitted.select(['.*_fitted']),{'opacity':0},'Fitted CCDC',True)


# Synthetic composites visualizing
# Take common false color composite bands and visualize them for the next to the last year

# First get the bands of predicted bands and then split off the name
fittedBns = fitted.select(['.*_fitted']).first().bandNames()
bns = fittedBns.map(lambda bn: ee.String(bn).split('_').get(0))

# Filter down to the next to the last year and a summer date range
compositeYear = endYear-1
syntheticComposites = fitted.select(fittedBns,bns)\
    .filter(ee.Filter.calendarRange(compositeYear,compositeYear,'year'))
    # .filter(ee.Filter.calendarRange(190,250)).first()

# Visualize output as you would a composite
getImagesLib.vizParamsFalse['dateFormat']='YY-MM-dd'
getImagesLib.vizParamsFalse['advanceInterval']='day'
Map.addTimeLapse(syntheticComposites,getImagesLib.vizParamsFalse,f'Synthetic Composite Time Lapse {compositeYear}')

Map.turnOnInspector()
Map.view()

#No layers will draw, but you can double click on map to see the fitted CCDC time series

Adding layer: Fitted CCDC
Adding layer: Synthetic Composite Time Lapse 2021
Starting webmap
Using default refresh token for geeView
Local web server at: http://localhost:1231/geeView/ already serving.
cwd a:\GEE\gee_py_modules_package\geeViz\examples
geeView URL: http://localhost:1231/geeView/?projectID=lcms-292214&accessToken=ya29.a0AeDClZAfStv0wHfQ1B_cHdomQAvSXnIkthwptm1W3wSHeeXGip11eZ9ukuCTShfb-ZGggYwemvX73S887-cqFqtNV_odRSd_iApHU4yDRrOxSkUWzuL8DvaPOlMxNslQfW9Ct7ztFIL_ewIIzi6cDbDowBhtMMfpDU0pYNBIoiUaCgYKAWQSARESFQHGX2Mi-kD5cbYGXXP4O5Kwr0E_mA0178

Combining CCDC outputs#

CCDC is expensive to run. It also runs out of memory when run over many observations. Since it initializes each pixel with global values, in its current form, it is not easy to pick up an existing CCDC output and extend it further in time.

In order to avoid completely re-running CCDC every year, starting in 2023, we developed a method to combine two overlapping CCDC runs into a single time series otuput

This method uses a basic linearly weighted feathering techinque. Coefficients from two overlapping runs are averaged with a linear weighting from 1-0 and 0-1 for the early and late outputs respectively over a specified overlapping period

The example below will demonstrate how to combine two CCDC outputs

Map.clearMap()
# Bring in second collection
ccdcBandNames = [
    "tStart",
    "tEnd",
    "tBreak",
    "changeProb",
    'swir1.*',
    "NDVI.*",
]
ccdcImg1 = ee.ImageCollection("projects/lcms-292214/assets/CONUS-LCMS/Base-Learners/CCDC-Collection-1984-2022").select(ccdcBandNames).mosaic()

ccdcImg2 = ee.ImageCollection("projects/lcms-292214/assets/CONUS-LCMS/Base-Learners/CCDC-Feathered-Collection").select(ccdcBandNames).mosaic()


# Set years to include union of all years of the two CCDC outputs
startYear = 1984
endYear = 2024

# Important parameters - when to feather the two together
# Has to fall within the overlapping period of the two runs
# In general, the longer the period, the better. 
# Keeping it away from the very first and very last year of either of the runs is a good idea
featheringStartYear = 2014
featheringEndYear = 2021

# Set a date range and date step (proportion of year - 1 = annual, 0.1 = 10 images per year)
# 245 is a good startJulian and endJulian if step = 1. Set startJulian to 1 and endJulian to 365 and step to 0.1 to see seasonality
startJulian = 245
endJulian = 245
step = 1

# Choose which band to show
fitted_band = "NDVI_CCDC_fitted"


# Get fitted for early, late, and combined
# The predictCCDC function will automatically feather two raw CCDC images together if two are provided
# If a single image is provided, no feathering will be performed
timeImgs = changeDetectionLib.simpleGetTimeImageCollection(startYear, endYear, startJulian, endJulian, step)
fittedFeathered =changeDetectionLib.predictCCDC([ccdcImg1,ccdcImg2],timeImgs,fillGaps,whichHarmonics,featheringStartYear,featheringEndYear)
fittedEarly =changeDetectionLib.predictCCDC(ccdcImg1,timeImgs,fillGaps,whichHarmonics)
fittedLate =changeDetectionLib.predictCCDC(ccdcImg2,timeImgs,fillGaps,whichHarmonics)

# Give each unique band names
fittedFeathered = fittedFeathered.select([fitted_band],[f'{fitted_band}_Combined'])
fittedEarly = fittedEarly.select([fitted_band],[f'{fitted_band}_Early'])
fittedLate = fittedLate.select([fitted_band],[f'{fitted_band}_Late'])

# Join all 3
joined = fittedEarly.linkCollection(fittedLate,[f'{fitted_band}_Late'],None,'system:time_start')
joined = joined.linkCollection(fittedFeathered,[f'{fitted_band}_Combined'],None,'system:time_start')

# Show on map
Map.addLayer(joined, {'reducer':ee.Reducer.mean(),'min':0.3,'max':0.8}, "Combined CCDC", True)

Map.turnOnInspector()
Map.view()

Adding layer: Combined CCDC
Starting webmap
Using default refresh token for geeView
Local web server at: http://localhost:1231/geeView/ already serving.
cwd a:\GEE\gee_py_modules_package\geeViz\examples
geeView URL: http://localhost:1231/geeView/?projectID=lcms-292214&accessToken=ya29.a0AeDClZBh2uA3pK6W_QM5fBogIflJRhg25ewKoA3xsaR0OKftli_0AWaSveD2MuiMLrHlCC6ZpXiWFqGMihCwO5b8_GZeCZPh_3guT3Sl5GrfKS_mqn9ooW1rPx0RC8S4hDgcuU3_2rXmVvizKWDv_b1lXrZcP_tc3ggiDUxWuAwaCgYKARQSARESFQHGX2Mi31I_AAI_7LPZrWZ78tsVHg0178

Combining CCDC Outputs for Change Detection#

Two CCDC outputs can also be combined for change detection The two outputs are concatenated, with the most recent and most probable from either input considered for change.

Both inputs much have non-null values for a pixel to be considered for change detection. Any pixel with a null value in either input image will result in that pixel always being null for change.

Map.clearMap()


changeObjCombined = changeDetectionLib.ccdcChangeDetection([ccdcImg1,ccdcImg2],changeDetectionBandName,startYear,endYear)
s
Map.addLayer(changeObjCombined[sortingMethod]['loss']['year'],{'min':startYear,'max':endYear,'palette':changeDetectionLib.lossYearPalette},'Loss Year Combined')
Map.addLayer(changeObjCombined[sortingMethod]['loss']['mag'],{'min':-0.5,'max':-0.1,'palette':changeDetectionLib.lossMagPalette},'Loss Mag Combined',False)
Map.addLayer(changeObjCombined[sortingMethod]['gain']['year'],{'min':startYear,'max':endYear,'palette':changeDetectionLib.gainYearPalette},'Gain Year Combined')
Map.addLayer(changeObjCombined[sortingMethod]['gain']['mag'],{'min':0.05,'max':0.2,'palette':changeDetectionLib.gainMagPalette},'Gain Mag Combined',False)


Map.turnOnInspector()
Map.view()

Adding layer: Loss Year Combined
Adding layer: Loss Mag Combined
Adding layer: Gain Year Combined
Adding layer: Gain Mag Combined
Starting webmap
Using default refresh token for geeView
Local web server at: http://localhost:8001/geeView/ already serving.
cwd a:\GEE\gee_py_modules_package\geeViz\examples
geeView URL: http://localhost:8001/geeView/?projectID=lcms-292214&accessToken=ya29.a0AeDClZBr9o4QCr0_2wM6o36L_3Tp36ZFWxRDSmjai_9ChFEFQSNwYcwe3MB3M29gzBl228iWzrZQaCtGJLA-BYj0j1tYyzs-Y7OgTU7qgEZL-IacVbQx8FQUCNNU4bJeQAnF0x-Huf2rtDFFD1qcVjet1hvEl1KAGnMLEoNt9y8aCgYKAdQSARESFQHGX2MiFyLq6iGmT8gP2wJ-KmZ-dQ0178