Model Calibration

Summary

The purpose of this webpage is to describe the data, methods, and results associated with the development of satellite-based water quality models for High Rock Lake, NC. This work was supported by the North Carolina Attorney General’s Office Environmental Enhancement Grant Program (Award: WFU021PRE1), and a research team of faculty and students from Wake Forest University (WFU), North Carolina Agricultural and Technical State University (NCAT), and North Carolina Central University (NCCU). Our goal was to develop a low-cost and comprehensive water quality monitoring system for High Rock Lake that can inform ongoing nutrient load reduction efforts across the watershed. While this study focuses on High Rock Lake, the approach can be applied to other lakes. The calibrated models described here are applied to Sentinel-2 satellite imagery in Google Earth Engine (GEE) applications that allow users to analyze time series and view multiple maps of Chl-a and TSS for any point or region of the lake. The in-situ data we collected is available through the High Rock Lake Water Quality Dashboard, which contains additional watershed-scale water quality information and summative maps derived from multi-temporal satellite imagery.

1.0 Data Collection and Processing

1.1 Overview 

We collected satellite/in-situ match-ups on 19 dates spread over 2021, 2022, and 2023; grab samples and hyperspectral reflectance measurements were collected on the same day as a Sentinel-2 satellite overpass. Our sampling locations within the lake are shown in Figure 1. We identified 9 sampling sites in the main lake section that align with stations the state used in past monitoring and modeling studies, and added 3 additional stations in the Town Crane Creek arm, as we noticed the water in this section appeared very green, indicating high Chl-a concentrations. We used the ArcGIS Survey 123 mobile phone application to record the time and location of our sampling locations. All samples were collected within 3 hours of the satellite overpass.

1.2 Laboratory Analysis of Grab Samples

Dr. Kyana Young (WFU Engineering) led the analysis of grab samples and trained several undergraduate and graduate students in the development and implementation of protocols. We measured Chl-a, TSS, and Color Dissolved Organic Matter (CDOM). We followed the EPA 445 method (Arar and Collins, 1997) for the Chl-a protocol and used a Trilogy Fluorometer with a non-acidification module. We used the APHA-AWWA-WEF (2005) method for TSS, and the Ocean Optics protocol for CDOM (Mueller et al., 2003). All samples were stored in a cooler and processed within 24 hours of collection. On each sampling date, we collected triplicates of one of the stations, allowing us to calculate the standard deviation of the in-situ Chl-a measurement.

1.3 Hyperspectral Reflectance

Hyperspectral reflectance measurements serve as an in-situ measure of remote sensing reflectance and can be compared to satellite reflectance measurements to improve satellite data processing methods and to assess the relative error magnitude. The in-situ reflectance was collected with the WISP-3 handheld above-water radiometer. This device has 3 sensors that measure downwelling radiance at a 45 deg angle directed at the sky, upwelling radiance at a 45 deg angle directed toward the water surface, and downwelling irradiance from above (all directions). Measurements from these 3 sensors can be combined to estimate remote sensing reflectance, using Eq 1 (Mobley, 1999). We collected 5 readings at each sampling site. Individual readings were uploaded to the WISP web, and latitude and longitude information from the Survey 123 data were added by matching time stamps. The data was exported as a csv file and duplicate data and outlier reflectance spectral were removed before averaging the measurements for each location. The remote sensing reflectance spectra were then weighted by the spectral response functions for the Sentinel-2 bands and aggregated to obtain values that have the same radiometric resolution as the Senitnel-2 bands. The in-situ measured hyperspectral reflectance of a sampling point is shown in Figure 3, along with the aggregated reflectance that aligns with the Sentinel-2 bands.

1.4 Satellite Image Processing

The Sentinel-2 images acquired on field sampling dates were downloaded from the Sentinel Hub EO (Earth Observing) Browser (https://www.sentinel-hub.com/explore/eobrowser/). The images were atmospherically corrected, where the top-of-atmosphere reflectance values are transformed to bottom-of-atmosphere reflectance values, using ACOLITE (version 20210802.0), a program that uses a dark spectrum fitting approach (Vanhellemont & Ruddick, 2016; Vanhellemont, 2019, 2020) that is appropriate for inland and coastal waters. The ACOLITE parameter settings were modified to include sun glint corrections and a fixed path for the aerosol optical thickness retrieval, based on the relatively small size of the study area. Ancillary data, including elevation from the SRTM DEM, was pulled into ACOLITE through an EarthData account. This algorithm also masks out cloudy pixels. An example of a cloud-free Sentinel-2 image for High Rock Lake before and after the ACOLITE atmospheric correction is applied is shown in Figure 4.

While ACOLITE is a widely used atmospheric correction algorithm, it is not applied operationally and must be applied by individual users on images of interest. This is not ideal for developing an operational tool that does not require manual updates. Therefore, we also evaluated an atmospheric correction algorithm implemented in Google Earth Engine (GEE) (Page et al., 2019). GEE is a powerful cloud environment that stores large amounts of satellite data, including all Sentinel-2 imagery, and contains a suite of data processing and modeling tools that can be applied to images to produce customized end-user products. After applying the atmospheric correction algorithm, we exported the images and extracted data associated with in-situ sampling points so we could explore and calibrate water quality models offline.

2.0  Statistical Modeling and Error Analysis

After all data processing and cleaning steps were completed, we had a total sample size of 166 satellite/in-situ match-ups that could be used for model development and calibration. We used this data to calibrate two models: one that estimates Chl-a from remote sensing reflectance, and one that estimates TSS from remote sensing reflectance. 

2.1 Chl-a Model

We tested several alternative model structures, but here we present the final model that we selected. Studies have shown that performance can be improved if different models are used for different optical water types within a water body (Moore et al., 2014; Uudeberg et al., 2020).  As seen in Figure 4, the upper reach of High Rock Lake is where the Yadkin River enters the lake and is very turbid (indicating high TSS) compared to the main stem and lake arms that have high Chl-a levels. We therefore split the 3 sampling locations in the upper reach from the other sites and used the model shown in Eq 2 for the turbid sites and the model shown in Eq 3 for all other sites. Eq. 2 fits a quadratic function to the Normalized Difference Chlorophyll Index (NDCI) (Mishra and Mishra, 2012) and uses the red-edge 1 (λ  = 705 nm) and red (λ  = 665 nm) bands. These wavelengths are known to capture the fluorescence signal in the spectra. A simpler model that is a linear function of the ratio of the red-edge 1 band to the red band (Eq 3) performed well for all other stations, below the upstream section. 

The model performance is summarized in Figures 5 and 6 and Table 1, and the final model with fitted coefficients is provided in Equations 4 and 5. Figure 5 shows the in-situ versus predicted Chl-a values for the aggregated WISP-3 values, the Sentinel-2 pixels atmospherically corrected with ACOLITE, and the Sentinel-2 pixels atmospherically corrected with the GEE algorithm.  Figure 6 shows the residual plots for the Sentinel-2 GEE model that was selected and used in the operational tool, showing that errors are balanced but not quite normally distributed (probabilities in the tails are higher) and variance is generally constant with Chl-a magnitude.

Key error metrics that were used for model selection and analysis are provided in Table 1, and they were calculated using Equations 6 ­– 8, showing the coefficient of determination (R-squared), mean absolute error (MAE), and bias. The bias formulation is based on Seagers et al. (2018) recommendation for optical models of Chl-a, where values greater than unity suggest that the model over-predicts Chl-a. All of the models have a slight positive bias, meaning that they overpredict Chl-a by up to 5%. Seagers et al. also recommended using a log transformed MAE metric, based on the large dataset used in their study where in-situ Chl-a values varied by an order of magnitude. However, this is not the case with our dataset, so we opted for the standard MAE formulation. MAE is considered the most appropriate error metric for a model that is designed for prediction, the intended use of our model. However, we used the GEE atmospheric correction algorithm in our final application due to its integration in GEE, allowing for a more useful operational product.

The WISP-3 model is shown for comparison, representing a model that is less sensitive to errors and uncertainties from the satellite atmospheric correction algorithm and the spatial variability within a 20-meter satellite pixel. While the WISP-3 performs better (as expected) than the Sentinel-2 models, significant variability remains that we attribute to uncertainty in the Chl-a laboratory analysis our grab samples. Based on our triplicate grab samples, the 90% Confidence interval of our in-situ Chl-a is 29.21 µg L-1. We used leave-one-out cross validation to produce a 90% confidence interval for the GEE Sentinel-2 Chl-a model, and it was slightly smaller, 27.99 µg L-1. According to the NC Department of Water Resources 2019 – 2022 Chl-a round robin reports, which evaluate the consistency and uncertainty of Chl-a protocols across several labs, the aggregated 90% confidence interval is 22.90 µg L-1, which is smaller than our comparable satellite model estimate but still significant. Based on this analysis, we conclude the performance of our satellite model is acceptable in the context of other existing uncertainties. 

2.2 TSS Model

Development and calibration of the TSS model was more straightforward as we already evaluated several alternative satellite-based models for TSS at High Rock Lake using historical data and Landsat-7 imagery. The published results (Di Vittorio et al., 2023) demonstrate that a semi-analytical non-linear model that is a function of a single band (Nechad et al., 2010) had the best overall performance. The model formulation is provided in Eq 9. A WFU student (Aidan Norris) calibrated this model with different Sentinel-2 bands in our satellite/in-situ match-up dataset (n = 166). The red (λ  = 665 nm) and red-edge (λ  = 705 nm) bands showed much stronger correlations with TSS compared to other bands, so models were calibrated with both bands. The optimal models are shown in Figure 7 and the error metrics compared for selection are provided in Table 2. The final model with the fitted coefficients is provided in Eq. 10.

The error metrics for the two alternative models include the R-squared (Eq. 4), MAE (Eq. 5), and mean bias (MB; Eq. 8). The bias metric used for the Chl-a model (Eq. 6) was designed for Chl-a data, so a more traditional bias metric was used here, where a value of 0 indicates no bias. Because this is a non-linear model, we used brute-force optimization to identify the optimal coefficients with an error function equal to the sum of MAE and MB, the same approach was used in our published work (Di Vittorio et al., 2023). MAE was used to determine the best model, and the red-edge (λ  = 705 nm) band was selected.

3.0  Implementation in GEE Web Applications and Water Quality Dashboard

The calibrated Chl-a and TSS models were integrated into two different GEE web applications and an ArcGIS dashboard. WFU students Michael Moerk and Julia Mascialino led the GEE application development, and NCCU students Michael Berryman, Marlene, and David led the development of the dashboard, with guidance from Dr. Chris Zarzar and Dr. Rakesh Malhotra. These tools are designed for end users and allow them to easily interact with the satellite-based maps.

The GEE applications contain maps and time series data for all Sentinel-2 imagery, not just the dates used for calibration, and automatically updates as new images are acquired. As of Nov. 12, 2025, 588 images of High Rock Lake were available. The first application (Fig. 8) allows users to select either a Chl-a or TSS map for the most recent Sentinel-2 image and then select a point or draw a rectangle or polygon from which time series TSS and Chl-a data are extracted, averaged, and plotted.  Users can subsequently download the time series data as a table. In the second application (Fig. 9), users can compare maps side-by-slide with an interactive slider and select from a true color image, a Chl-a map, or a TSS map. Users can also select specific dates for each panel, enabling change comparisons in addition to water quality parameter comparisons.

To implement the statistical models for TSS and Chl-a, we had to make a few adjustments to the optimal calibrated models. With regards to the Chl-a model, to implement the dual-model formulation, we would need to develop a way to first classify a pixel by the optical water type before applying the most appropriate model. This would require additional research and development that we did not have the time to implement. We therefore decided to only implement the linear model (Eq. 5) that applies to the significantly larger portion of the lake. This model more accurately captures high Chl-a values that are of particular interest to stakeholders. With regards to the TSS model we set an upper limit of 50 mg/L. Due to the nonlinear structure of the TSS model (Eq. 10), when remote sensing reflectance is high, a very small increase in reflectance results in a very large increase in TSS. We selected 50 as the upper limit to align with the highest in-situ TSS measurements, in our dataset and historical data collected by the state. This is a conservative approach and accounts for the fact that we were not able to validate the model accuracy for values greater than 50.

In response to feedback we received from staff at the NC Department of Water Resources (DWR), we also created some summative satellite-based water quality maps that aggregate the large multi-temporal dataset that is available on the GEE map and added them to our ArcGIS water quality dashboard. We exported the full time series of Chl-a maps from July 2016 to August 2024 and calculated monthly median maps that were subsequently imported as 12 individual maps into the dashboard. The maps were clipped to the lake extents with a 100 meter interior buffer applied to eliminate edge pixels that are likely impacted by vegetation along the lake boundaries (producing inflated Chl-a estimates). These maps allow users to assess seasonal patterns of Chl-a values across different portions of the lake. Figure 10 provides a screenshot of one of the dashboard panels that includes the September monthly median map. Potential point and non-point source nutrient and sediment loads are also shown in this view, with the goal of helping users conceptualize the distribution of pollutant sources across the watershed and explore associates with the lake water quality and how they may vary in space and time.

Finally, Figure 11 provides examples of other spatiotemporal analyses that can be done with the satellite-based water quality maps. Because Chl-a is being used for assessing the lake nutrient impairment status, we extracted exceedance probability curves for individual locations across different sections of the lake (top images) and an aggregate map of the Chl-a geomean over time (bottom images; time range spans from July 2016 to Aug 2024 and includes 520 images) and show the 35 µg/L threshold that is considered the impairment threshold.

Acknowledgements

The research was funded by the N.C. Attorney General's Office Environmental Enhancement Grant Program (award WFU021PRE1). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the N.C. Attorney General's Office. We would like to thank staff at the N.C. Department of Water Resources for their feedback on the data visualization tools, including Mark Vander Borgh, Elizabeth Liebeg, Joey Hester, Nora Deamer, Virginia Baker, Elizabeth Kountis, Tammy Hill, and Ling Ren. We would also like to thank the Yadkin Riverkeepers, including Edgar Miller, Grace Fuchs, Bailey Hill, Stephanie Stephens, and Nicole Eastman for their feedback and collaboration throughout this project.

Attribution of Work Effort

Dr. Courtney Di Vittorio served as the project lead and supported all aspects of the project. Michael Moerk developed and tested the Chl-a model, and Aidan Norris developed and tested the TSS model. Julia Mascialino supported the development of the Google Earth Engine applications, Jaime Cardenas, Amelia Suhocki, Michael Huang, Jacob Louie, Esteban Coyoy Cifuentes, and Abby Kneisel supported the laboratory analysis of grab samples, Dr. Chris Zarzar, Michael Moerk, Bailey Hill, Melita Wiles, Jaime Cardenas, Eden Wasehun, Michael Berryman, Edgar Miller, Lily Hezrony, Evelyn Lorenzo, and Isabela Gonzalez supported field sampling campaigns. Dr. Chris Zarzar, Dr. Rakesh Malhotra, Michael Berryman, David Bjorkback, and Marlene Perrin developed the ArcGIS Enterprise data portal.

References

APHA-AWWA-WEF (2005) Standard Methods for the Examination of Water and Wastewater. 21th Edition. New York, Total Solids Suspended, Method 2540 D, 2-55 a 2-59.

Arar, E.J. and G.B. Collins. (1997). Method 445.0: In vitro determination of chlorophyll a and pheophytin a in marine and freshwater algae by fluorescence. Revision 1.2. United States Environmental Protection Agency, National Exposure Research Laboratory

Di Vittorio, C. A., Moerk, M., & Kreutzberger, W. (2023). Enhancing perspectives on lake impairments using satellite observations: A case study on High Rock Lake, North Carolina. JAWRA Journal of the American Water Resources Association, 59(5), 1067-1083.

Mishra, S., & Mishra, D. R. (2012). Normalized difference chlorophyll index: A novel model for remote estimation of chlorophyll-a concentration in turbid productive waters. Remote Sensing of Environment, 117, 394-406.

Mobley, C. D. (1999) "Estimation of the remote-sensing reflectance from above-surface measurements," Appl. Opt. 38, 7442-7455

Moore, T. S., Dowell, M. D., Bradt, S., & Verdu, A. R. (2014). An optical water type framework for selecting and blending retrievals from bio-optical algorithms in lakes and coastal waters. Remote sensing of environment, 143, 97-111.

Mueller, J. L., Fargion, G. S., & McClain, C. R. (Eds.). (2003). Ocean Optics Protocols for Satellite Ocean Color Sensor Validation, Revision 4, Volume IV: Inherent Optical Properties: Instruments, Characterizations, Field Measurements and Data Analysis Protocols

Nechad, B., Ruddick, K. G., & Park, Y. (2010). Calibration and validation of a generic multisensor algorithm for mapping of total suspended matter in turbid waters. Remote Sensing of Environment, 114(4), 854-866.

Page, B. P., Olmanson, L. G., & Mishra, D. R. (2019). A harmonized image processing workflow using Sentinel-2/MSI and Landsat-8/OLI for mapping water clarity in optically variable lake systems. Remote Sensing of Environment, 231, 111284.

Uudeberg, K., Aavaste, A., Kõks, K. L., Ansper, A., Uusõue, M., Kangro, K., ... & Reinart, A. (2020). Optical water type guided approach to estimate optical water quality parameters. Remote Sensing, 12(6), 931.

Seegers, B. N., Stumpf, R. P., Schaeffer, B. A., Loftin, K. A., & Werdell, P. J. (2018). Performance metrics for the assessment of satellite data products: an ocean color case study. Optics express, 26(6), 7404-7422.

Vanhellemont, Q., and K. Ruddick. 2016. “Acolite for Sentinel-2: Aquatic Applications of MSI Imagery.” In Proceedings of the 2016 ESA Living Planet Symposium 9–13. Prague: ESA Special Publication SP-740.

Vanhellemont, Q. 2019. “Adaptation of the Dark Spectrum Fitting Atmospheric Correction for Aquatic Applications of the Landsat and Sentinel-2 Archives.” Remote Sensing of Environment 225: 175–92. https://doi.org/10.1016/j.rse.2019.03.010.

Vanhellemont, Q. 2020. “Sensitivity Analysis of the Dark Spectrum Fitting Atmospheric Correction for Metre-and Decametre-Scale Satellite Imagery Using Autonomous Hyperspectral Radiometry.” Optics Express 28(20): 29948–65. https://doi.org/10.1364/OE.397456.