43. GOES-16: True Color Images from GOES ABI#
Download the goes_true_color.ipynb notebook from the week8 folder
43.1. Introduction#
This is a modified version of Brian Blaylock’s UCAR python gallery notebook. Blaylock is the author of the GOES download package goes2go used below. I’ve made some changes to the notebook to expand the explanations and fix some bugs caused by changes to the data format and cartopy.
The notebook shows how to make a true color image from the GOES-16 Advanced Baseline Imager (ABI) level 2 data. We will plot the image with matplotlib and Cartopy. The image can be displayed on any map projection after applying a transformation.
Some background: Take a look at the 16 channels described on page 230 of Stull Chapter 8. Note that unlike Modis or Landsat OLI, the ABI doesn’t have a visible channel at green wavelengths, substituting the near-ir 0.846–0.885 \(\mu m\) wavelength range for green. Since that channel is very responsive to plant chlorophyll it’s possible to still use that as part of a proxy for green as shown below. As of March 2025 GOES 18 is GOES West, and GOES 16 is GOES East, with GOES 17 moved into a parking position and GOES 19 ready to replace GOES 16 in Aparil. For more background on GOES, see NOAA’s beginner guide to GOES.
43.1.1. Current GOES satellites#
There are three active GOES satellite on station as of February, 2025. See the NESDIS website for a full list.
GOES 16 is GOES East, parked at -75 degrees West longitude over the equator
GOES 18 is GOES West, parked at -136.9 degrees West longitude
GOES 19 will replace GOES 16 in April, 2025
We will be using the Advanced Baseline Imager Level 2 Cloud and Moisture Product from the Amazon AWS GOES repository. The full list of available GOES products on AWS is here.
The file we’ll download is domain ‘C’ of the “Cloud Moisture Imagery” dataset – abbreviated ABI-L2-MCMIPC
which expands to “Advanced Baseline Imager level 2 Cloud Moisture Imagery Product for the
Continental US”. It’s about 68 Mbytes, compared to the full disk file (ABI-L2-MCMIPC domain F) which
is about 305 Mbytes. A folder called ~/repos/a301/satdata/goes will be created to hold the download.
43.2. Channels and workflow#
These are the channels that contribute to the true-color composite:
– |
Wavelength |
Channel |
Description |
|---|---|---|---|
Red |
0.64 m |
2 |
Red Visible |
Green |
0.86 m |
3 |
Veggie Near-IR |
Blue |
0.47 m |
1 |
Blue Visible |
The workflow for the notebook:
Find a scene using goes2go
Read the metadata and channels into an xarray dataset
Create the cartopy crs for the scene using the xarray metpy plugin.
Get the
original_extentfor the image in the geostationary projection crs so we can plot itProduce a weighted “pseudo-green” image using a weighted combination of the 3 bands to replace the missing green band.
Clip the band values to 0-1 and apply a “gamma correction” (an alternative to the histogram equalization we used in Making color composite (“false color”) images)
Stack the 3 bands in rgb order using numpy depth stacking
Plot the mapped image using cartopy
Zoom the image by changing the axis extent
from goes2go.data import goes_nearesttime
import xarray
from datetime import datetime, timedelta
from pathlib import Path
import cartopy
import numpy as np
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import cartopy.feature as cfeature
from pathlib import Path
import warnings
warnings.filterwarnings('ignore')
/Users/phil/mini310/envs/a301/lib/python3.13/site-packages/goes2go/data.py:673: FutureWarning: 'H' is deprecated and will be removed in a future version. Please use 'h' instead of 'H'.
within=pd.to_timedelta(config["nearesttime"].get("within", "1h")),
/Users/phil/mini310/envs/a301/lib/python3.13/site-packages/goes2go/NEW.py:185: FutureWarning: 'H' is deprecated and will be removed in a future version. Please use 'h' instead of 'H'.
within=pd.to_timedelta(config["nearesttime"].get("within", "1h")),
43.3. Read in the data#
NOAA stores the data in a file format called netcdf instead of geotiff. Rioxarray isn’t able to read
the current netcdf version of these files, but plain xarray works. The goes2go library loads an xarray plugin
calle metpy that is able to read the crs information from the netcdf metadata, which is stored in
set of attributes at the_xarray.goes_imager_projection.attrs. We metpy.parse_cf function to read
those attributes and return a cartopy crs for mapping.
The 16 bands in the dataset are labeled CMI_C01, CMI_C02 etc.
Each band has data quality flag file DQF_C01, DQF_C02, …a band wavelength data stored as band_wavelength_C01, ..., and a band id stored as band_id_C01, ....
43.3.1. Find the file on AWS using goes_nearesttime#
The data is stored on Amazon S3 and can
be downloaded via an https link given by goes_nearresttime
help(goes_nearesttime)
Help on function goes_nearesttime in module goes2go.data:
goes_nearesttime(
attime,
within=Timedelta('0 days 01:00:00'),
*,
satellite='noaa-goes16',
product='ABI-L2-MCMIP',
domain='C',
return_as='xarray',
download=True,
overwrite=False,
save_dir=PosixPath('/Users/phil/data'),
bands=None,
s3_refresh=True,
ignore_missing=None,
verbose=True
)
Get the GOES data nearest a specified time.
Parameters
----------
attime : datetime
Time to find the nearest observation for.
May also use a pandas-interpretable datetime string.
within : timedelta or pandas-parsable timedelta str
Timerange tht the nearest observation must be.
satellite : {'goes16', 'goes17', 'goes18'}
Specify which GOES satellite.
The following alias may also be used:
- ``'goes16'``: 16, 'G16', or 'EAST'
- ``'goes17'``: 17, 'G17', or 'WEST'
- ``'goes18'``: 18, 'G18', or 'WEST'
product : {'ABI', 'GLM', other GOES product}
Specify the product name.
- 'ABI' is an alias for ABI-L2-MCMIP Multichannel Cloud and Moisture Imagery
- 'GLM' is an alias for GLM-L2-LCFA Geostationary Lightning Mapper
Others may include ``'ABI-L1b-Rad'``, ``'ABI-L2-DMW'``, etc.
For more available products, look at this `README
<https://docs.opendata.aws/noaa-goes16/cics-readme.html>`_
domain : {'C', 'F', 'M'}
ABI scan region indicator. Only required for ABI products if the
given product does not end with C, F, or M.
- C: Contiguous United States (alias 'CONUS')
- F: Full Disk (alias 'FULL')
- M: Mesoscale (alias 'MESOSCALE')
return_as : {'xarray', 'filelist'}
Return the data as an xarray.Dataset or as a list of files
download : bool
- True: Download the data to disk to the location set by :guilabel:`save_dir`
- False: Just load the data into memory.
save_dir : pathlib.Path or str
Path to save the data.
overwrite : bool
- True: Download the file even if it exists.
- False: Do not download the file if it already exists
bands : None, int, or list
ONLY FOR L1b-Rad products; specify the bands you want
s3_refresh : bool
Refresh the s3fs.S3FileSystem object when files are listed.
save_dir = Path.home() / "repos/a301/satdata/goes"
43.3.1.1. Emoji bug!#
This version of jupyter notebooks and/or goes2go triggers a pretty obscure bug. A successful read prints out info with a couple of emoji to make the output look pretty. Unfortunately, jupyter notebook is unable to save the notebook once the cell includes those emoji. So you need to execute the cell below twice: first to get the data with getdata=True and then again to make the emoji go away with getdata=False
getdata = False
if getdata:
g = goes_nearesttime(
datetime(2020, 6, 25, 18), satellite="goes16",product="ABI-L2-MCMIP", domain='C',
return_as="xarray", save_dir = save_dir, download = True, overwrite = False
)
the_path = g.path[0]
else:
the_path = ("noaa-goes16/ABI-L2-MCMIPC/2020/177/18/"
"OR_ABI-L2-MCMIPC-M6_G16_s20201771801172_e20201771803551_c20201771804104.nc")
full_path = save_dir / the_path
This example uses the level 2 multiband formatted file for the continental US (C) domain
Here’s the pattern used for the file names:
OR_ABI-L2-MCMIPC-M3_G16_s20181781922189_e20181781924562_c20181781925075.nc
OR - Indicates the system is operational
ABI - Instrument type
L2 - Level 2 Data
MCMIP - Multichannel Cloud and Moisture Imagery products
C - CONUS file (created every 5 minutes) or F (full disk)
M3 - Scan mode
G16 - GOES-16
s##### - Scan start: 4 digit year, 3 digit day of year (Julian day), hour, minute, second, tenth second
e##### - Scan end
c##### - File Creation
.nc - NetCDF file extension
43.3.2. open file and read crs#
goesC = xarray.open_dataset(full_path,mode = 'r',mask_and_scale = True)
len(goesC.variables)
161
There are 161 different variables, here are the first 10 – the channels and their data quality flags:
goesC.x
<xarray.DataArray 'x' (x: 2500)> Size: 10kB
array([-0.101332, -0.101276, -0.10122 , ..., 0.0385 , 0.038556, 0.038612],
shape=(2500,), dtype=float32)
Coordinates:
t datetime64[ns] 8B ...
* x (x) float32 10kB -0.1013 -0.1013 -0.1012 ... 0.0385 0.03856 0.03861
y_image float32 4B ...
x_image float32 4B ...
Attributes:
units: rad
axis: X
long_name: GOES fixed grid projection x-coordinate
standard_name: projection_x_coordinategoesC.coords['x']
<xarray.DataArray 'x' (x: 2500)> Size: 10kB
array([-0.101332, -0.101276, -0.10122 , ..., 0.0385 , 0.038556, 0.038612],
shape=(2500,), dtype=float32)
Coordinates:
t datetime64[ns] 8B ...
* x (x) float32 10kB -0.1013 -0.1013 -0.1012 ... 0.0385 0.03856 0.03861
y_image float32 4B ...
x_image float32 4B ...
Attributes:
units: rad
axis: X
long_name: GOES fixed grid projection x-coordinate
standard_name: projection_x_coordinatelist(goesC.variables.keys())[:10]
['CMI_C01',
'DQF_C01',
'CMI_C02',
'DQF_C02',
'CMI_C03',
'DQF_C03',
'CMI_C04',
'DQF_C04',
'CMI_C05',
'DQF_C05']
43.3.3. read the crs from band 2#
The cartopy crs information is associated with individual variables, grab it from band 2 (arbitrary choice)
band_2 = goesC.metpy.parse_cf('CMI_C02')
cartopy_crs = band_2.metpy.cartopy_crs
cartopy_crs.to_wkt()
'PROJCRS["unknown",BASEGEOGCRS["unknown",DATUM["Unknown based on GRS 1980 ellipsoid",ELLIPSOID["GRS 1980",6378137,298.2572221,LENGTHUNIT["metre",1,ID["EPSG",9001]]]],PRIMEM["Greenwich",0,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8901]]],CONVERSION["unknown",METHOD["Geostationary Satellite (Sweep X)"],PARAMETER["Longitude of natural origin",-75,ANGLEUNIT["degree",0.0174532925199433],ID["EPSG",8802]],PARAMETER["Satellite Height",35786023,LENGTHUNIT["metre",1,ID["EPSG",9001]]],PARAMETER["False easting",0,LENGTHUNIT["metre",1],ID["EPSG",8806]],PARAMETER["False northing",0,LENGTHUNIT["metre",1],ID["EPSG",8807]]],CS[Cartesian,2],AXIS["(E)",east,ORDER[1],LENGTHUNIT["metre",1,ID["EPSG",9001]]],AXIS["(N)",north,ORDER[2],LENGTHUNIT["metre",1,ID["EPSG",9001]]],REMARK["PROJ CRS string: +proj=geos +a=6378137.0 +rf=298.2572221 +lon_0=-75.0 +lat_0=0.0 +h=35786023.0 +x_0=0 +y_0=0 +units=m +sweep=x +no_defs"]]'
import pyproj
test = pyproj.CRS.from_user_input(cartopy_crs.to_wkt())
test.to_json()
'{"$schema":"https://proj.org/schemas/v0.7/projjson.schema.json","type":"ProjectedCRS","name":"unknown","base_crs":{"name":"unknown","datum":{"type":"GeodeticReferenceFrame","name":"Unknown based on GRS 1980 ellipsoid","ellipsoid":{"name":"GRS 1980","semi_major_axis":6378137,"inverse_flattening":298.2572221}},"coordinate_system":{"subtype":"ellipsoidal","axis":[{"name":"Latitude","abbreviation":"lat","direction":"north","unit":"degree"},{"name":"Longitude","abbreviation":"lon","direction":"east","unit":"degree"}]}},"conversion":{"name":"unknown","method":{"name":"Geostationary Satellite (Sweep X)"},"parameters":[{"name":"Longitude of natural origin","value":-75,"unit":"degree","id":{"authority":"EPSG","code":8802}},{"name":"Satellite Height","value":35786023,"unit":"metre"},{"name":"False easting","value":0,"unit":"metre","id":{"authority":"EPSG","code":8806}},{"name":"False northing","value":0,"unit":"metre","id":{"authority":"EPSG","code":8807}}]},"coordinate_system":{"subtype":"Cartesian","axis":[{"name":"Easting","abbreviation":"E","direction":"east","unit":"metre"},{"name":"Northing","abbreviation":"N","direction":"north","unit":"metre"}]},"remarks":"PROJ CRS string: +proj=geos +a=6378137.0 +rf=298.2572221 +lon_0=-75.0 +lat_0=0.0 +h=35786023.0 +x_0=0 +y_0=0 +units=m +sweep=x +no_defs"}'
Here are the y coordinates in the geostationary projection
band_2.y
<xarray.DataArray 'y' (y: 1500)> Size: 12kB
array([4588197.756227, 4586193.790336, 4584189.824445, ..., 1588183.762427,
1586179.796536, 1584175.830645], shape=(1500,))
Coordinates:
t datetime64[ns] 8B ...
y_image float32 4B 0.08624
x_image float64 8B -1.122e+06
* y (y) float64 12kB 4.588e+06 4.586e+06 ... 1.586e+06 1.584e+06
metpy_crs object 8B Projection: geostationary
Attributes:
units: meter
axis: Y
long_name: GOES fixed grid projection y-coordinate
standard_name: projection_y_coordinate43.4. Get the x,y extent and start time#
original_extent=(band_2.x.data.min(), band_2.x.data.max(), band_2.y.data.min(), band_2.y.data.max())
original_extent
(np.float64(-3626269.332309611),
np.float64(1381769.9431296065),
np.float64(1584175.830644913),
np.float64(4588197.756226748))
goesC.time_coverage_start
'2020-06-25T18:01:17.2Z'
43.5. Get Date and Time Information#
Each file represents the data collected during one scan sequence for the domain. There are several different time stamps in this file, which are also found in the file’s name.
# Scan's start time, converted to datetime object
scan_start = datetime.strptime(goesC.time_coverage_start, "%Y-%m-%dT%H:%M:%S.%fZ")
# Scan's end time, converted to datetime object
scan_end = datetime.strptime(goesC.time_coverage_end, "%Y-%m-%dT%H:%M:%S.%fZ")
# File creation time, convert to datetime object
file_created = datetime.strptime(goesC.date_created, "%Y-%m-%dT%H:%M:%S.%fZ")
#
# arithmetic on datetime objects
#
duration = (scan_end - scan_start).seconds / 60
midpoint = scan_start + timedelta(minutes = duration/2.)
print(f"Scan Start : {scan_start}")
print(f"Scan End : {scan_end}")
print(f"File Created : {file_created}")
print(f"Scan midpoint : {midpoint}")
Scan Start : 2020-06-25 18:01:17.200000
Scan End : 2020-06-25 18:03:55.100000
File Created : 2020-06-25 18:04:10.400000
Scan midpoint : 2020-06-25 18:02:35.700000
43.6. True Color Recipe#
43.6.1. Gamma correction#
Color images are a Red-Green-Blue (RGB) composite of three different channels. We will assign the following channels as our RGB values:
RGB values must be between 0 and 1, same as the range of values of the reflectance channels. A gamma correction is applied to control the brightness and make the image not look too dark, where for input and output values brightness values V the gamma correction is:
This correction will enhance the distance between smaller values ov \(V_{in}\) (i.e. more levels for darker colors)
To see why this is, suppose \(\gamma = 2\), so \(1/\gamma\)=0.5. Take the derivative of (43.1):
so the smaller the magnitude of \(V_{in}\), the larger the difference between \(dV_{out}\) and \(dV_{in}\)
Most displays have a decoding gamma of 2.2. See wikipedia, and this tutorial if you’d like to know more.
43.6.2. Natural vs. veggie green#
The GREEN “veggie” channel on GOES-16 corresponds to the landsat NIR band 5. We could use that channel in place of green, but it would make the green in our image appear too vibrant. Instead, we will tone-down the green channel by interpolating the value to simulate a natural green color.
or, a simple alternative (CIMSS Natural True Color):
The multiband formatted file we loaded is convenient because all the GOES channels are in the same NetCDF file. Next, we will assign our variables R, G, and B as the data for each channel.
43.7. Check the band wavelengths#
Read the band_wavelength keys to double check the wavelengths for each of bands
2, 3 and 1.
# Confirm that each band is the wavelength we are interested in
for band in [2, 3, 1]:
#
# create the key and use it to grabe the name, wavelength and units
#
band_wavelength = f"band_wavelength_C{band:02d}"
print(band_wavelength)
long_name = goesC[band_wavelength].long_name
wavelength = goesC[band_wavelength].data[0]
units = goesC[band_wavelength].units
print(f"{long_name} is {wavelength:.2f} {units}")
band_wavelength_C02
ABI band 2 central wavelength is 0.64 um
band_wavelength_C03
ABI band 3 central wavelength is 0.87 um
band_wavelength_C01
ABI band 1 central wavelength is 0.47 um
a=goesC["CMI_C02"]
a.x
<xarray.DataArray 'x' (x: 2500)> Size: 10kB
array([-0.101332, -0.101276, -0.10122 , ..., 0.0385 , 0.038556, 0.038612],
shape=(2500,), dtype=float32)
Coordinates:
t datetime64[ns] 8B ...
* x (x) float32 10kB -0.1013 -0.1013 -0.1012 ... 0.0385 0.03856 0.03861
y_image float32 4B 0.08624
x_image float32 4B -0.03136
Attributes:
units: rad
axis: X
long_name: GOES fixed grid projection x-coordinate
standard_name: projection_x_coordinate43.8. Clip the bands and apply the gamma correction#
######################################################################
#
# Load the three channels into appropriate R, G, and B variables
R = goesC["CMI_C02"].data
G = goesC["CMI_C03"].data
B = goesC["CMI_C01"].data
######################################################################
#
# Apply range limits for each channel. RGB values must be between 0 and 1
R = np.clip(R, 0, 1)
G = np.clip(G, 0, 1)
B = np.clip(B, 0, 1)
######################################################################
#
# Apply a gamma correction to the image
gamma = 2.2
R = np.power(R, 1 / gamma)
G = np.power(G, 1 / gamma)
B = np.power(B, 1 / gamma)
######################################################################
43.9. Make “true” green from the three bands#
# Calculate the "True" Green
G_true = 0.45 * R + 0.1 * G + 0.45 * B
G_true = np.maximum(G_true, 0)
G_true = np.minimum(G_true, 1)
43.10. Plot the raw images#
First, we plot each channel individually. The deeper the color means the satellite is observing more light in that channel. Clouds appear white becuase they reflect lots of red, green, and blue light. You will also notice that the land reflects a lot of “green” in the veggie channel becuase this channel is sensitive to the chlorophyll.
fig, ([ax1, ax2, ax3, ax4]) = plt.subplots(1, 4, figsize=(16, 3))
ax1.imshow(R, cmap="Reds", vmax=1, vmin=0)
ax1.set_title("Red", fontweight="semibold")
ax1.axis("off")
ax2.imshow(G, cmap="Greens", vmax=1, vmin=0)
ax2.set_title("Veggie", fontweight="semibold")
ax2.axis("off")
ax3.imshow(G_true, cmap="Greens", vmax=1, vmin=0)
ax3.set_title('"True" Green', fontweight="semibold")
ax3.axis("off")
ax4.imshow(B, cmap="Blues", vmax=1, vmin=0)
ax4.set_title("Blue", fontweight="semibold")
ax4.axis("off")
plt.subplots_adjust(wspace=0.02)
43.11. Stack the tree images using dstack#
Compare a stack with “veggie green” with “true green” using false color images
######################################################################
# The addition of the three channels results in a color image. We combine the
# three channels in a stacked array and display the image with `imshow` again.
#
# The RGB array with the raw veggie band
RGB_veggie = np.dstack([R, G, B])
# The RGB array for the true color image
RGB = np.dstack([R, G_true, B])
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(16, 6))
# The RGB using the raw veggie band
ax1.imshow(RGB_veggie)
ax1.set_title("GOES-16 RGB Raw Veggie", fontweight="semibold", loc="left", fontsize=12)
ax1.set_title("%s" % scan_start.strftime("%d %B %Y %H:%M UTC "), loc="right")
ax1.axis("off")
# The RGB for the true color image
ax2.imshow(RGB)
ax2.set_title("GOES-16 RGB True Color", fontweight="semibold", loc="left", fontsize=12)
ax2.set_title("%s" % scan_start.strftime("%d %B %Y %H:%M UTC "), loc="right")
ax2.axis("off");
43.12. Plot the mapped image using cartopy#
fig = plt.figure(figsize=(15, 12))
ax = fig.add_subplot(1, 1, 1, projection=cartopy_crs)
ax.imshow(
RGB,
origin="upper",
extent= original_extent,
transform=cartopy_crs,
interpolation="nearest",
vmin=162.0,
vmax=330.0,
)
ax.coastlines(resolution="50m", color="black", linewidth=2)
ax.add_feature(ccrs.cartopy.feature.STATES)
plt.title("GOES-16 True Color", loc="left", fontweight="semibold", fontsize=15)
plt.title("%s" % scan_start.strftime("%d %B %Y %H:%M UTC "), loc="right");
43.13. Changing the map projection and plot extent#
The next two cells show how to change the plotting crs and boundaries (extent) on a plot
The plot below changes the crs to LambertConformal and sets the plotting extent to between -130- -60 deg lon and 10 - 65 deg lat
fig = plt.figure(figsize=(15, 12))
lc = ccrs.LambertConformal(central_longitude=-97.5)
#
# add an axiss with the lc projection
#
ax = fig.add_subplot(1, 1, 1, projection=lc)
#
# lat/lons are given in "flat-square" projection
#
ax.set_extent([-120, -65, 10, 55], crs=ccrs.PlateCarree())
ax.imshow(
RGB,
origin="upper",
extent=original_extent,
transform=cartopy_crs,
interpolation="none",
)
ax.coastlines(resolution="50m", color="black", linewidth=1)
ax.add_feature(ccrs.cartopy.feature.STATES)
plt.title("GOES-16 True Color", loc="left", fontweight="semibold", fontsize=15)
plt.title("%s" % scan_start.strftime("%d %B %Y %H:%M UTC "), loc="right");
43.14. Change the axis extent (not the original image extent)#
We can zoom the map to a lon/lat box by specifying a new extent for the axis between -114.5 - -108.25 deg Lon and 36-43 deg Lat – Salt Lake City, Utah The axis crs is switched to PlateCarree, which is ok at this small scale (where the lat/lon parallels are basically straight lines)
fig = plt.figure(figsize=(8, 8))
#
# GOES
#
pc = ccrs.PlateCarree()
ax = fig.add_subplot(1, 1, 1, projection=pc)
#
# utah
#
ax.set_extent([-114.75, -108.25, 36, 43], crs=pc)
ax.imshow(RGB,
origin='upper',
extent=original_extent,
transform=cartopy_crs,
interpolation='none')
ax.coastlines(resolution='50m', color='black', linewidth=1)
ax.add_feature(ccrs.cartopy.feature.STATES)
plt.title('GOES-16 True Color', loc='left', fontweight='bold', fontsize=15)
plt.title('{}'.format(scan_start.strftime('%d %B %Y %H:%M UTC ')), loc='right');
43.15. Practice questions (takehome exam prep)#
43.15.1. Practice question 1#
Use a seaborn jointplot to compare the channel 1 (blue) histogram before and after the gamma correction
43.15.2. Practice question 2#
Use xarray.isel to clip just the portion of the abi scene that’s in the [-114.75, -108.25, 36, 43] lon/lat bounding box and save it to disk as a netcdf file with the correct affine transform and crs
43.15.3. Practice question 3#
Change the map projection in from lambert to azimuthal equal area – does it look less weird?
43.15.4. Practice question 4#
Write a channel out as a geotiff file by first converting it to a rioxarray
43.15.5. Pratice question 5#
Use the full domain image to produce a picture of a region in the southern hemisphere or Canada