Using 'open virtual dataset' capability to work with TEMPO Level 3 data¶
Summary¶
In this tutorial, we will use the earthaccess.open_virtual_mfdataset() function to open a week's worth of granules from the Nitrogen Dioxide (NO2) Level-3 data collection of the TEMPO air quality mission.
About TEMPO: The Tropospheric Emissions: Monitoring of Pollution (TEMPO) instrument is a geostationary satellite mission that provides hourly daytime measurements of air quality over North America. It measures key pollutants including nitrogen dioxide (NO2), formaldehyde, and ozone at high spatial resolution (~2 by 4.75 km at the center of its field of regard).
We will calculate temporal and spatial means for a subset of the data and visualize the results. This approach demonstrates cloud-optimized data access patterns that can scale from days to years of data.
Learn more: For comprehensive documentation on the earthaccess package, visit the earthaccess documentation.
Note that this same approach can be used for a date range of any length, within the mission's duration. Running this notebook for a year's worth of TEMPO Level-3 data took approximately 15 minutes.
Prerequisites¶
AWS US-West-2 Environment: This tutorial has been designed to run in an AWS cloud compute instance in AWS region us-west-2. However, if you want to run it from your laptop or workstation, everything should work just fine but without the speed benefits of in-cloud access.
Earthdata Account: A (free!) Earthdata Login account is required to access data from the NASA Earthdata system. Before requesting TEMPO data, we first need to set up our Earthdata Login authentication, as described in the Earthdata Cookbook's earthaccess tutorial (link).
Packages:
cartopydaskearthaccessversion 0.14.0 or greatermatplotlibnumpyxarray
Setup¶
In [1]:
Copied!
import cartopy.crs as ccrs
import earthaccess
import matplotlib.pyplot as plt
import numpy as np
import xarray as xr
from matplotlib import rcParams
%config InlineBackend.figure_format = 'jpeg'
rcParams["figure.dpi"] = (
80 # Reduce figure resolution to keep the saved size of this notebook low.
)
import cartopy.crs as ccrs
import earthaccess
import matplotlib.pyplot as plt
import numpy as np
import xarray as xr
from matplotlib import rcParams
%config InlineBackend.figure_format = 'jpeg'
rcParams["figure.dpi"] = (
80 # Reduce figure resolution to keep the saved size of this notebook low.
)
Login using the Earthdata Login¶
In [2]:
Copied!
auth = earthaccess.login()
if not auth.authenticated:
# Ask for credentials and persist them in a .netrc file.
auth.login(strategy="interactive", persist=True)
print(earthaccess.__version__)
auth = earthaccess.login()
if not auth.authenticated:
# Ask for credentials and persist them in a .netrc file.
auth.login(strategy="interactive", persist=True)
print(earthaccess.__version__)
0.15.1
Search for data granules¶
We search for TEMPO Nitrogen Dioxide (NO2) data for a week-long period (note: times are in UTC), between January 11th and 18th, 2024. We are limiting the results to render the notebook quickly.
In [3]:
Copied!
results = earthaccess.search_data(
# TEMPO NO₂ Level-3 product
short_name="TEMPO_NO2_L3",
# Version 3 of the data product
version="V03",
# Time period: One week in January 2024 (times are in UTC)
temporal=("2024-01-11 12:00", "2024-01-18 12:00"),
count=8,
)
print(f"Number of granules found: {len(results)}")
results = earthaccess.search_data(
# TEMPO NO₂ Level-3 product
short_name="TEMPO_NO2_L3",
# Version 3 of the data product
version="V03",
# Time period: One week in January 2024 (times are in UTC)
temporal=("2024-01-11 12:00", "2024-01-18 12:00"),
count=8,
)
print(f"Number of granules found: {len(results)}")
Number of granules found: 8
Opening Virtual Multifile Datasets¶
Understanding Virtual Datasets¶
Virtual datasets allow us to work with multiple files as if they were a single dataset without downloading all the data to local storage. This is achieved through:
- Kerchunk: Creates lightweight reference files that point to data chunks in cloud storage
- Virtualizarr: Combines multiple reference files into a single virtual dataset
- Lazy Loading: Data is only accessed when needed for computations
For TEMPO data, we need to handle the hierarchical netCDF4 structure by opening each group separately, then merging them.
First we set the argument options to be used by earthaccess.open_virtual_mfdataset.
load argument considerations:
load=Trueworks. Withinearthaccess.open_virtual_mfdataset, a temporary virtual reference file (a "virtual dataset") is created and then immediately loaded with kerchunk. This is because the function assumes the user is making this request for the first time and the combined manifest file needs to be generated first. In the future, however,earthaccess.open_virtual_mfdatasetmay provide a way to save the combined manifest file, at which point you could then avoid repeating these steps, and proceed directly to loading with kerchunk/virtualizarr.load=Falseresults inKeyError: "no index found for coordinate 'longitude'"because it createsManifestArrays without indexes (see the earthaccess documentation here (link))
In [4]:
Copied!
open_options = {
"access": "indirect", # access to cloud data (faster in AWS with "direct")
"load": True, # Load metadata immediately (required for indexing)
"concat_dim": "time", # Concatenate files along the time dimension
"data_vars": "minimal", # Only load data variables that include the concat_dim
"coords": "minimal", # Only load coordinate variables that include the concat_dim
"compat": "override", # Avoid coordinate conflicts by picking the first
"combine_attrs": "override", # Avoid attribute conflicts by picking the first
}
open_options = {
"access": "indirect", # access to cloud data (faster in AWS with "direct")
"load": True, # Load metadata immediately (required for indexing)
"concat_dim": "time", # Concatenate files along the time dimension
"data_vars": "minimal", # Only load data variables that include the concat_dim
"coords": "minimal", # Only load coordinate variables that include the concat_dim
"compat": "override", # Avoid coordinate conflicts by picking the first
"combine_attrs": "override", # Avoid attribute conflicts by picking the first
}
Because TEMPO data are processed and archived in a netCDF4 format using a group hierarchy, we open each group – i.e., 'root', 'product', and 'geolocation' – and then afterwards merge them together.
In [5]:
Copied!
%%time
result_root = earthaccess.open_virtual_mfdataset(granules=results, **open_options)
result_product = earthaccess.open_virtual_mfdataset(
granules=results, group="product", **open_options
)
result_geolocation = earthaccess.open_virtual_mfdataset(
granules=results, group="geolocation", **open_options
)
# merge
result_merged = xr.merge([result_root, result_product, result_geolocation])
result_merged
%%time
result_root = earthaccess.open_virtual_mfdataset(granules=results, **open_options)
result_product = earthaccess.open_virtual_mfdataset(
granules=results, group="product", **open_options
)
result_geolocation = earthaccess.open_virtual_mfdataset(
granules=results, group="geolocation", **open_options
)
# merge
result_merged = xr.merge([result_root, result_product, result_geolocation])
result_merged
CPU times: user 375 ms, sys: 87.5 ms, total: 462 ms Wall time: 34.6 s
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[5], line 1 ----> 1 get_ipython().run_cell_magic('time', '', 'result_root = earthaccess.open_virtual_mfdataset(granules=results, **open_options)\nresult_product = earthaccess.open_virtual_mfdataset(\n granules=results, group="product", **open_options\n)\nresult_geolocation = earthaccess.open_virtual_mfdataset(\n granules=results, group="geolocation", **open_options\n)\n# merge\nresult_merged = xr.merge([result_root, result_product, result_geolocation])\nresult_merged\n') File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2565, in InteractiveShell.run_cell_magic(self, magic_name, line, cell) 2563 with self.builtin_trap: 2564 args = (magic_arg_s, cell) -> 2565 result = fn(*args, **kwargs) 2567 # The code below prevents the output from being displayed 2568 # when using magics with decorator @output_can_be_silenced 2569 # when the last Python token in the expression is a ';'. 2570 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False): File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1452, in ExecutionMagics.time(self, line, cell, local_ns) 1450 if interrupt_occured: 1451 if exit_on_interrupt and captured_exception: -> 1452 raise captured_exception 1453 return 1454 return out File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1416, in ExecutionMagics.time(self, line, cell, local_ns) 1414 st = clock2() 1415 try: -> 1416 exec(code, glob, local_ns) 1417 out = None 1418 # multi-line %%time case File <timed exec>:1 File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/earthaccess/dmrpp_zarr.py:188, in open_virtual_mfdataset(granules, group, access, preprocess, parallel, load, reference_dir, reference_format, **xr_combine_nested_kwargs) 182 vmfdataset.virtualize.to_kerchunk(str(ref_), format=reference_format) 184 storage_options = { 185 "remote_protocol": fs.protocol, 186 "remote_options": fs.storage_options, 187 } --> 188 vds = xr.open_dataset( 189 str(ref_), 190 engine="kerchunk", 191 storage_options=storage_options, 192 ) 193 return vds 195 return vmfdataset File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/xarray/backends/api.py:596, in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, create_default_indexes, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs) 584 decoders = _resolve_decoders_kwargs( 585 decode_cf, 586 open_backend_dataset_parameters=backend.open_dataset_parameters, (...) 592 decode_coords=decode_coords, 593 ) 595 overwrite_encoded_chunks = kwargs.pop("overwrite_encoded_chunks", None) --> 596 backend_ds = backend.open_dataset( 597 filename_or_obj, 598 drop_variables=drop_variables, 599 **decoders, 600 **kwargs, 601 ) 602 ds = _dataset_from_backend_dataset( 603 backend_ds, 604 filename_or_obj, (...) 615 **kwargs, 616 ) 617 return ds File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/kerchunk/xarray_backend.py:13, in KerchunkBackend.open_dataset(self, filename_or_obj, storage_options, open_dataset_options, **kw) 9 def open_dataset( 10 self, filename_or_obj, *, storage_options=None, open_dataset_options=None, **kw 11 ): 12 open_dataset_options = (open_dataset_options or {}) | kw ---> 13 ref_ds = open_reference_dataset( 14 filename_or_obj, 15 storage_options=storage_options, 16 open_dataset_options=open_dataset_options, 17 ) 18 return ref_ds File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/kerchunk/xarray_backend.py:45, in open_reference_dataset(filename_or_obj, storage_options, open_dataset_options) 42 if open_dataset_options is None: 43 open_dataset_options = {} ---> 45 store = refs_as_store(filename_or_obj, **storage_options) 47 return xr.open_zarr( 48 store, zarr_format=2, consolidated=False, **open_dataset_options 49 ) File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/kerchunk/utils.py:53, in refs_as_store(refs, read_only, fs, remote_protocol, remote_options, **kwargs) 50 remote_options = remote_options or {} 51 remote_options["asynchronous"] = True ---> 53 fss = refs_as_fs( 54 refs, 55 fs=fs, 56 remote_protocol=remote_protocol, 57 remote_options=remote_options, 58 **kwargs, 59 ) 60 return fs_as_store(fss, read_only=read_only) File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/kerchunk/utils.py:34, in refs_as_fs(refs, fs, remote_protocol, remote_options, asynchronous, **kwargs) 32 remote_options = remote_options.copy() if remote_options else {} 33 remote_options["asynchronous"] = asynchronous ---> 34 fs = fsspec.filesystem( 35 "reference", 36 fo=refs, 37 fs=fs, 38 remote_protocol=remote_protocol, 39 remote_options=remote_options, 40 **kwargs, 41 asynchronous=asynchronous, 42 ) 43 return fs File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/fsspec/registry.py:322, in filesystem(protocol, **storage_options) 315 warnings.warn( 316 "The 'arrow_hdfs' protocol has been deprecated and will be " 317 "removed in the future. Specify it as 'hdfs'.", 318 DeprecationWarning, 319 ) 321 cls = get_filesystem_class(protocol) --> 322 return cls(**storage_options) File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/fsspec/spec.py:84, in _Cached.__call__(cls, *args, **kwargs) 82 return cls._cache[token] 83 else: ---> 84 obj = super().__call__(*args, **kwargs, **strip_tokenize_options) 85 # Setting _fs_token here causes some static linters to complain. 86 obj._fs_token_ = token File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/fsspec/implementations/reference.py:767, in ReferenceFileSystem.__init__(self, fo, target, ref_storage_args, target_protocol, target_options, remote_protocol, remote_options, fs, template_overrides, simple_templates, max_gap, max_block, cache_size, **kwargs) 764 break 766 if remote_protocol and remote_protocol not in self.fss: --> 767 fs = filesystem(remote_protocol, **(remote_options or {})) 768 self.fss[remote_protocol] = fs 770 self.fss[None] = fs or filesystem("file") # default one File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/fsspec/registry.py:321, in filesystem(protocol, **storage_options) 314 if protocol == "arrow_hdfs": 315 warnings.warn( 316 "The 'arrow_hdfs' protocol has been deprecated and will be " 317 "removed in the future. Specify it as 'hdfs'.", 318 DeprecationWarning, 319 ) --> 321 cls = get_filesystem_class(protocol) 322 return cls(**storage_options) File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/fsspec/registry.py:258, in get_filesystem_class(protocol) 256 if protocol not in registry: 257 if protocol not in known_implementations: --> 258 raise ValueError(f"Protocol not known: {protocol}") 259 bit = known_implementations[protocol] 260 try: ValueError: Protocol not known: ('http', 'https')
In [6]:
Copied!
result_merged = xr.merge([result_root, result_product, result_geolocation])
result_merged
result_merged = xr.merge([result_root, result_product, result_geolocation])
result_merged
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[6], line 1 ----> 1 result_merged = xr.merge([result_root, result_product, result_geolocation]) 2 result_merged NameError: name 'result_root' is not defined
Understanding the Data¶
- vertical_column_troposphere: Total column amount of NO₂ in the troposphere (units: molecules/cm²)
- main_data_quality_flag: Quality indicator (0 = good quality data)
- Geographic region: We'll focus on the Mid-Atlantic region (Washington DC area)
- Longitude: -78° to -74° W
- Latitude: 35° to 39° N
In [7]:
Copied!
# Define our region of interest (Mid-Atlantic/Washington DC area)
lon_bounds = (-78, -74) # Western to Eastern longitude
lat_bounds = (35, 39) # Southern to Northern latitude
print(
f"Analyzing region: {lat_bounds[0]}°N to {lat_bounds[1]}°N, {abs(lon_bounds[0])}°W to {abs(lon_bounds[1])}°W"
)
# Define our region of interest (Mid-Atlantic/Washington DC area)
lon_bounds = (-78, -74) # Western to Eastern longitude
lat_bounds = (35, 39) # Southern to Northern latitude
print(
f"Analyzing region: {lat_bounds[0]}°N to {lat_bounds[1]}°N, {abs(lon_bounds[0])}°W to {abs(lon_bounds[1])}°W"
)
Analyzing region: 35°N to 39°N, 78°W to 74°W
Temporal Mean - a map showing an annual average¶
In [8]:
Copied!
# Define temporal mean (average over time) calculation
temporal_mean_ds = (
result_merged.sel(
{
"longitude": slice(lon_bounds[0], lon_bounds[1]),
"latitude": slice(lat_bounds[0], lat_bounds[1]),
}
)
.where(result_merged["main_data_quality_flag"] == 0) # Filter for good quality data
.mean(dim="time")
)
print(f"Dataset shape after subsetting: {temporal_mean_ds.dims}")
temporal_mean_ds
# Define temporal mean (average over time) calculation
temporal_mean_ds = (
result_merged.sel(
{
"longitude": slice(lon_bounds[0], lon_bounds[1]),
"latitude": slice(lat_bounds[0], lat_bounds[1]),
}
)
.where(result_merged["main_data_quality_flag"] == 0) # Filter for good quality data
.mean(dim="time")
)
print(f"Dataset shape after subsetting: {temporal_mean_ds.dims}")
temporal_mean_ds
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[8], line 3 1 # Define temporal mean (average over time) calculation 2 temporal_mean_ds = ( ----> 3 result_merged.sel( 4 { 5 "longitude": slice(lon_bounds[0], lon_bounds[1]), 6 "latitude": slice(lat_bounds[0], lat_bounds[1]), 7 } 8 ) 9 .where(result_merged["main_data_quality_flag"] == 0) # Filter for good quality data 10 .mean(dim="time") 11 ) 13 print(f"Dataset shape after subsetting: {temporal_mean_ds.dims}") 14 temporal_mean_ds NameError: name 'result_merged' is not defined
In [9]:
Copied!
%%time
# Compute the temporal mean
mean_vertical_column_trop = temporal_mean_ds["vertical_column_troposphere"].compute()
%%time
# Compute the temporal mean
mean_vertical_column_trop = temporal_mean_ds["vertical_column_troposphere"].compute()
CPU times: user 6 μs, sys: 1e+03 ns, total: 7 μs Wall time: 8.58 μs
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[9], line 1 ----> 1 get_ipython().run_cell_magic('time', '', '# Compute the temporal mean\nmean_vertical_column_trop = temporal_mean_ds["vertical_column_troposphere"].compute()\n') File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2565, in InteractiveShell.run_cell_magic(self, magic_name, line, cell) 2563 with self.builtin_trap: 2564 args = (magic_arg_s, cell) -> 2565 result = fn(*args, **kwargs) 2567 # The code below prevents the output from being displayed 2568 # when using magics with decorator @output_can_be_silenced 2569 # when the last Python token in the expression is a ';'. 2570 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False): File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1452, in ExecutionMagics.time(self, line, cell, local_ns) 1450 if interrupt_occured: 1451 if exit_on_interrupt and captured_exception: -> 1452 raise captured_exception 1453 return 1454 return out File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1416, in ExecutionMagics.time(self, line, cell, local_ns) 1414 st = clock2() 1415 try: -> 1416 exec(code, glob, local_ns) 1417 out = None 1418 # multi-line %%time case File <timed exec>:2 NameError: name 'temporal_mean_ds' is not defined
In [10]:
Copied!
fig, ax = plt.subplots(subplot_kw={"projection": ccrs.PlateCarree()})
mean_vertical_column_trop.squeeze().plot.contourf(ax=ax)
# Add geographic features
ax.coastlines()
ax.gridlines(
draw_labels=True,
dms=True,
x_inline=False,
y_inline=False,
)
plt.show()
fig, ax = plt.subplots(subplot_kw={"projection": ccrs.PlateCarree()})
mean_vertical_column_trop.squeeze().plot.contourf(ax=ax)
# Add geographic features
ax.coastlines()
ax.gridlines(
draw_labels=True,
dms=True,
x_inline=False,
y_inline=False,
)
plt.show()
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[10], line 3 1 fig, ax = plt.subplots(subplot_kw={"projection": ccrs.PlateCarree()}) ----> 3 mean_vertical_column_trop.squeeze().plot.contourf(ax=ax) 5 # Add geographic features 6 ax.coastlines() NameError: name 'mean_vertical_column_trop' is not defined
Spatial mean - a time series of area averages¶
In [11]:
Copied!
# Define spatial mean (average over longitude/latitude) calculation
spatial_mean_ds = (
result_merged.sel(
{
"longitude": slice(lon_bounds[0], lon_bounds[1]),
"latitude": slice(lat_bounds[0], lat_bounds[1]),
}
)
.where(result_merged["main_data_quality_flag"] == 0) # Filter for good quality data
.mean(dim=("longitude", "latitude"))
)
spatial_mean_ds
# Define spatial mean (average over longitude/latitude) calculation
spatial_mean_ds = (
result_merged.sel(
{
"longitude": slice(lon_bounds[0], lon_bounds[1]),
"latitude": slice(lat_bounds[0], lat_bounds[1]),
}
)
.where(result_merged["main_data_quality_flag"] == 0) # Filter for good quality data
.mean(dim=("longitude", "latitude"))
)
spatial_mean_ds
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[11], line 3 1 # Define spatial mean (average over longitude/latitude) calculation 2 spatial_mean_ds = ( ----> 3 result_merged.sel( 4 { 5 "longitude": slice(lon_bounds[0], lon_bounds[1]), 6 "latitude": slice(lat_bounds[0], lat_bounds[1]), 7 } 8 ) 9 .where(result_merged["main_data_quality_flag"] == 0) # Filter for good quality data 10 .mean(dim=("longitude", "latitude")) 11 ) 12 spatial_mean_ds NameError: name 'result_merged' is not defined
In [12]:
Copied!
%%time
# Compute the spatial mean
spatial_mean_vertical_column_trop = spatial_mean_ds[
"vertical_column_troposphere"
].compute()
%%time
# Compute the spatial mean
spatial_mean_vertical_column_trop = spatial_mean_ds[
"vertical_column_troposphere"
].compute()
CPU times: user 5 μs, sys: 0 ns, total: 5 μs Wall time: 9.3 μs
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[12], line 1 ----> 1 get_ipython().run_cell_magic('time', '', '# Compute the spatial mean\nspatial_mean_vertical_column_trop = spatial_mean_ds[\n "vertical_column_troposphere"\n].compute()\n') File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2565, in InteractiveShell.run_cell_magic(self, magic_name, line, cell) 2563 with self.builtin_trap: 2564 args = (magic_arg_s, cell) -> 2565 result = fn(*args, **kwargs) 2567 # The code below prevents the output from being displayed 2568 # when using magics with decorator @output_can_be_silenced 2569 # when the last Python token in the expression is a ';'. 2570 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False): File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1452, in ExecutionMagics.time(self, line, cell, local_ns) 1450 if interrupt_occured: 1451 if exit_on_interrupt and captured_exception: -> 1452 raise captured_exception 1453 return 1454 return out File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1416, in ExecutionMagics.time(self, line, cell, local_ns) 1414 st = clock2() 1415 try: -> 1416 exec(code, glob, local_ns) 1417 out = None 1418 # multi-line %%time case File <timed exec>:2 NameError: name 'spatial_mean_ds' is not defined
In [13]:
Copied!
spatial_mean_vertical_column_trop.plot()
plt.show()
spatial_mean_vertical_column_trop.plot()
plt.show()
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[13], line 1 ----> 1 spatial_mean_vertical_column_trop.plot() 2 plt.show() NameError: name 'spatial_mean_vertical_column_trop' is not defined
Single scan subset¶
In [14]:
Copied!
# Select a single scan time for detailed analysis
scan_time_start = np.datetime64("2024-01-11T13:00:00") # 1 PM UTC
scan_time_end = np.datetime64("2024-01-11T14:00:00") # 2 PM UTC
print(f"Analyzing single scan: {scan_time_start} to {scan_time_end} UTC")
print("Note: This corresponds to ~8-9 AM local time on the US East Coast")
subset_ds = result_merged.sel(
{
"longitude": slice(lon_bounds[0], lon_bounds[1]),
"latitude": slice(lat_bounds[0], lat_bounds[1]),
"time": slice(scan_time_start, scan_time_end),
}
).where(result_merged["main_data_quality_flag"] == 0)
subset_ds
# Select a single scan time for detailed analysis
scan_time_start = np.datetime64("2024-01-11T13:00:00") # 1 PM UTC
scan_time_end = np.datetime64("2024-01-11T14:00:00") # 2 PM UTC
print(f"Analyzing single scan: {scan_time_start} to {scan_time_end} UTC")
print("Note: This corresponds to ~8-9 AM local time on the US East Coast")
subset_ds = result_merged.sel(
{
"longitude": slice(lon_bounds[0], lon_bounds[1]),
"latitude": slice(lat_bounds[0], lat_bounds[1]),
"time": slice(scan_time_start, scan_time_end),
}
).where(result_merged["main_data_quality_flag"] == 0)
subset_ds
Analyzing single scan: 2024-01-11T13:00:00 to 2024-01-11T14:00:00 UTC Note: This corresponds to ~8-9 AM local time on the US East Coast
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[14], line 8 5 print(f"Analyzing single scan: {scan_time_start} to {scan_time_end} UTC") 6 print("Note: This corresponds to ~8-9 AM local time on the US East Coast") ----> 8 subset_ds = result_merged.sel( 9 { 10 "longitude": slice(lon_bounds[0], lon_bounds[1]), 11 "latitude": slice(lat_bounds[0], lat_bounds[1]), 12 "time": slice(scan_time_start, scan_time_end), 13 } 14 ).where(result_merged["main_data_quality_flag"] == 0) 15 subset_ds NameError: name 'result_merged' is not defined
In [15]:
Copied!
%%time
# Compute the single scan's values
subset_vertical_column_trop = subset_ds["vertical_column_troposphere"].compute()
%%time
# Compute the single scan's values
subset_vertical_column_trop = subset_ds["vertical_column_troposphere"].compute()
CPU times: user 6 μs, sys: 0 ns, total: 6 μs Wall time: 10 μs
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[15], line 1 ----> 1 get_ipython().run_cell_magic('time', '', '# Compute the single scan\'s values\nsubset_vertical_column_trop = subset_ds["vertical_column_troposphere"].compute()\n') File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/interactiveshell.py:2565, in InteractiveShell.run_cell_magic(self, magic_name, line, cell) 2563 with self.builtin_trap: 2564 args = (magic_arg_s, cell) -> 2565 result = fn(*args, **kwargs) 2567 # The code below prevents the output from being displayed 2568 # when using magics with decorator @output_can_be_silenced 2569 # when the last Python token in the expression is a ';'. 2570 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False): File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1452, in ExecutionMagics.time(self, line, cell, local_ns) 1450 if interrupt_occured: 1451 if exit_on_interrupt and captured_exception: -> 1452 raise captured_exception 1453 return 1454 return out File ~/checkouts/readthedocs.org/user_builds/earthaccess/envs/1144/lib/python3.11/site-packages/IPython/core/magics/execution.py:1416, in ExecutionMagics.time(self, line, cell, local_ns) 1414 st = clock2() 1415 try: -> 1416 exec(code, glob, local_ns) 1417 out = None 1418 # multi-line %%time case File <timed exec>:2 NameError: name 'subset_ds' is not defined
In [16]:
Copied!
fig, ax = plt.subplots(subplot_kw={"projection": ccrs.PlateCarree()})
subset_vertical_column_trop.squeeze().plot.contourf(ax=ax)
# Add geographic features
ax.coastlines()
ax.gridlines(
draw_labels=True,
dms=True,
x_inline=False,
y_inline=False,
)
plt.show()
fig, ax = plt.subplots(subplot_kw={"projection": ccrs.PlateCarree()})
subset_vertical_column_trop.squeeze().plot.contourf(ax=ax)
# Add geographic features
ax.coastlines()
ax.gridlines(
draw_labels=True,
dms=True,
x_inline=False,
y_inline=False,
)
plt.show()
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[16], line 3 1 fig, ax = plt.subplots(subplot_kw={"projection": ccrs.PlateCarree()}) ----> 3 subset_vertical_column_trop.squeeze().plot.contourf(ax=ax) 5 # Add geographic features 6 ax.coastlines() NameError: name 'subset_vertical_column_trop' is not defined
In [ ]:
Copied!