ECCOv4 access via NASA’s EarthData in the Cloud

This notebook demonstrates access to ECCOv4 model output. Broad information about the ECCO dataset can be found in the PODAAC website (see here).

Requirements to run this notebook

1. Have an Earth Data Login account

2. Have a Bearer Token.

Objectives

Use pydap’s client API to demonstrate

• Access to NASA’s EarthData via the use of tokens. Tokens are greatly favored over username/password, since tokens avoid repeated authentication over the many redirects.

• A workflow involving multiple OPeNDAP URLs and xarray parallelism, lazy evaluation, and plotting of Level 4 with complex Topology ECCOv4 Data via OPeNDAP.

Some variables of focus are

• Native grid

• Temperature and Salinity

Author: Miguel Jimenez-Urias, ‘24

import matplotlib.pyplot as plt
import numpy as np
from pydap.net import create_session
from pydap.client import open_url
import xarray as xr
import pydap

print("xarray version: ", xr.__version__)
print("pydap version: ", pydap.__version__)

xarray version:  2025.4.0
pydap version:  3.5.5

Access EARTHDATA

Many of the data variables can be browsed here. Here we will work with the original data defined on the Lat-Lon-Cap (LLC90) grid.

Grid_url = 'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Geometry%20Parameters%20for%20the%20Lat-Lon-Cap%2090%20(llc90)%20Native%20Model%20Grid%20(Version%204%20Release%204)/granules/GRID_GEOMETRY_ECCO_V4r4_native_llc0090'

Use .netrc credentials

my_session = create_session()

Alternative token approach

session_extra = {"token": "YourToken"}

# initialize a requests.session object with the token headers. All handled by pydap.
my_session = create_session(session_kwargs=session_extra)

Lazy access to remote data via pydap’s client API

pydap exploits the OPeNDAP’s separation between metadata and data, to create lazy dataset objects that point to the data. These lazy objects contain all the attributes detailed in OPeNDAP’s metadata files (DMR).

ds_grid = open_url(Grid_url, session=my_session, protocol="dap4")

ds_grid.tree()

.GRID_GEOMETRY_ECCO_V4r4_native_llc0090.nc
├──dxC
├──PHrefF
├──XG
├──dyG
├──rA
├──hFacS
├──Zp1
├──Zl
├──rAw
├──dxG
├──maskW
├──YC
├──XC
├──maskS
├──YG
├──hFacC
├──drC
├──drF
├──XC_bnds
├──Zu
├──Z_bnds
├──YC_bnds
├──PHrefC
├──rAs
├──Depth
├──dyC
├──SN
├──rAz
├──maskC
├──CS
├──hFacW
├──Z
├──i
├──i_g
├──j
├──j_g
├──k
├──k_l
├──k_p1
├──k_u
├──nb
├──nv
└──tile

Note

PyDAP accesses the remote dataset’s metadata, and no data has been downloaded yet!

Download data into memory

The syntax is as follows:

# this fetches remote data into a pydap object container
pydap_Array = dataset[<VarName>][:]

where <VarName> is the name of one of the variables in the pydap.model.BaseType object. The pydap.model.BaseType is a thing wrapper of numpy arrays. The array has been downloaded into “local” memory (RAM) as (uncompressed) numpy arrays.

To extract the data as a numpy array, run

# The `.data` command allows direct access to the Numpy array (e.g. for manipulation)
pydap_Array.data

# lets download some data
Depth = ds_grid['Depth'][:]
print(type(Depth))

<class 'pydap.model.BaseType'>

Depth.attributes

{'_FillValue': 9.969209968e+36,
 'long_name': 'model seafloor depth below ocean surface at rest',
 'units': 'm',
 'coordinate': 'XC YC',
 'coverage_content_type': 'modelResult',
 'standard_name': 'sea_floor_depth_below_geoid',
 'comment': "Model sea surface height (SSH) of 0m corresponds to an ocean surface at rest relative to the geoid. Depth corresponds to seafloor depth below geoid. Note: the MITgcm used by ECCO V4r4 implements 'partial cells' so the actual model seafloor depth may differ from the seafloor depth provided by the input bathymetry file.",
 'coordinates': 'YC XC',
 'origname': 'Depth',
 'fullnamepath': '/Depth',
 'Maps': (),
 'checksum': array([950678499], dtype=uint32)}

Depth.shape, Depth.dims

((13, 90, 90), ['/tile', '/j', '/i'])

Plot Depth along native grid

ECCO data is defined on a Cube Sphere – meaning the horizontal grid contains an extra dimension: tile or face. You can inspect the data in its native grid by plotting all horizontal data onto a grid as follows:

Variable = [Depth[i].data for i in range(13)]
clevels =  np.linspace(0, 6000, 100)
cMap = 'Greys_r'

fig, axes = plt.subplots(nrows=5, ncols=5, figsize=(8, 8), gridspec_kw={'hspace':0.01, 'wspace':0.01})
AXES = [
    axes[4, 0], axes[3, 0], axes[2, 0], axes[4, 1], axes[3, 1], axes[2, 1],
    axes[1, 1], 
    axes[1, 2], axes[1, 3], axes[1, 4], axes[0, 2], axes[0, 3], axes[0, 4],
]
for i in range(len(AXES)):
    AXES[i].contourf(Variable[i], clevels, cmap=cMap)

for ax in np.ravel(axes):
    ax.axis('off')
    plt.setp(ax.get_xticklabels(), visible=False)
    plt.setp(ax.get_yticklabels(), visible=False)

plt.show()

Fig. 1. Depth plotted on a horizontal layout. Data on tiles 0-5 follow C-ordering, whereas data on tiles 7-13 follow F-ordering. Data on the arctic cap, is defined on a polar coordinate grid.

Plot with corrected Topology

fig, axes = plt.subplots(nrows=4, ncols=4, figsize=(8, 8), gridspec_kw={'hspace':0.01, 'wspace':0.01})
AXES_NR = [
    axes[3, 0], axes[2, 0], axes[1, 0], axes[3, 1], axes[2, 1], axes[1, 1],
]
AXES_CAP = [axes[0, 0]]
AXES_R = [
    axes[1, 2], axes[2, 2], axes[3, 2], axes[1, 3], axes[2, 3], axes[3, 3],
]
for i in range(len(AXES_NR)):
    AXES_NR[i].contourf(Variable[i], clevels, cmap=cMap)

for i in range(len(AXES_CAP)):
    AXES_CAP[i].contourf(Variable[6].transpose()[:, ::-1], clevels, cmap=cMap)

for i in range(len(AXES_R)):
    AXES_R[i].contourf(Variable[7+i].transpose()[::-1, :], clevels, cmap=cMap)

for ax in np.ravel(axes):
    ax.axis('off')
    plt.setp(ax.get_xticklabels(), visible=False)
    plt.setp(ax.get_yticklabels(), visible=False)

plt.show()

Fig. 2. Depth plotted on a horizontal layout that approximates lat-lon display. Data on the arctic cap, however, remains on a polar coordinate grid.

pydap + xarray to aggregate multiple URLs

OPeNDAP allows remote access via dataURLs. Each dataURL represents a variable, i.e. a piece of the whole puzzle. We can exploit xarray concatenation and parallelism to combine multiple dataURLs, and thus multiple pydap objects, into a single xarray.Dataset. Can do this because xarray has long-implemented pydap as an engine to access/open remote datasets.

A single URL

Many OPeNDAP servers can serve data in either DAP2 and DAP4 (much newer) model implementations. In pydap, you can specify which of the two implementations by replacing the beginning of the URL with either one of dap2 or dap4. Since xarray imports pydap internally as is, then xarray can recognize this behavior as well.

Below we access remote Temperature and Salinity ECCO data with xarray via (internally) pydap.

baseURL = 'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/'
Temp_Salt = "ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_"
year = '2017-'
month = '01'
end_ = '_ECCO_V4r4_native_llc0090'
Temp_2017 = baseURL + Temp_Salt +year  + month + end_

Temp_2017

'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090'

Using CEs to pre-select only variables of interest

Allows the user to drop variables but ONLY after creating the dataset of the entire remote URL. With OPeNDAP servers, the you encode the names of the variables you want to access in the URL. THis can significantly speed up analysis and exploration.

Below we manually specify the DAP4 Constraint Expression for keeping only the variables THETA, along with its dimensions time, k, time, j, i and tile.

You can also construct the full URL with constraint expressions interactively, by selecting manually the variables on the datasets’s Data Request Form and selecting Copy (Raw) Data URL.

pyds = open_url(Temp_2017, session=my_session)
pyds.tree()

.OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.nc
├──XG
├──Zp1
├──Zl
├──YC
├──XC
├──SALT
├──YG
├──XC_bnds
├──Zu
├──THETA
├──Z_bnds
├──YC_bnds
├──time_bnds
├──Z
├──i
├──i_g
├──j
├──j_g
├──k
├──k_l
├──k_p1
├──k_u
├──nb
├──nv
├──tile
└──time

Pydap’s object can now helps up construct the Constraint Expression that will allow us to create a much simpler datacube

dims = pyds['THETA'].dims
Vars = ['/THETA'] + dims

# Below construct Contraint Expression
CE = "?dap4.ce="+(";").join(Vars)
print("Constraint Expression: ", CE)

Constraint Expression:  ?dap4.ce=/THETA;/time;/k;/tile;/j;/i

Now we add the CE to the data url as follows below:

Temp_2017 = baseURL + Temp_Salt +year  + month + end_ + CE
Temp_2017

'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090?dap4.ce=/THETA;/time;/k;/tile;/j;/i'

%%time
dataset = xr.open_dataset(Temp_2017, engine='pydap', session=my_session)
dataset

CPU times: user 151 ms, sys: 38.3 ms, total: 189 ms
Wall time: 30.5 s

<xarray.Dataset> Size: 21MB
Dimensions:  (time: 1, k: 50, tile: 13, j: 90, i: 90)
Coordinates:
  * i        (i) int32 360B 0 1 2 3 4 5 6 7 8 9 ... 81 82 83 84 85 86 87 88 89
  * j        (j) int32 360B 0 1 2 3 4 5 6 7 8 9 ... 81 82 83 84 85 86 87 88 89
  * k        (k) int32 200B 0 1 2 3 4 5 6 7 8 9 ... 41 42 43 44 45 46 47 48 49
  * tile     (tile) int32 52B 0 1 2 3 4 5 6 7 8 9 10 11 12
  * time     (time) datetime64[ns] 8B 2017-01-16T12:00:00
Data variables:
    THETA    (time, k, tile, j, i) float32 21MB ...
Attributes: (12/62)
    acknowledgement:                 This research was carried out by the Jet...
    author:                          Ian Fenty and Ou Wang
    cdm_data_type:                   Grid
    comment:                         Fields provided on the curvilinear lat-l...
    Conventions:                     CF-1.8, ACDD-1.3
    coordinates_comment:             Note: the global 'coordinates' attribute...
    ...                              ...
    time_coverage_duration:          P1M
    time_coverage_end:               2017-02-01T00:00:00
    time_coverage_resolution:        P1M
    time_coverage_start:             2017-01-01T00:00:00
    title:                           ECCO Ocean Temperature and Salinity - Mo...
    uuid:                            f5b7028c-4181-11eb-b7e6-0cc47a3f47b1

xarray.Dataset

• Dimensions:
  • time: 1
  • k: 50
  • tile: 13
  • j: 90
  • i: 90
• Coordinates: (5)
  • i
    (i)
    int32
    0 1 2 3 4 5 6 ... 84 85 86 87 88 89

    axis :

    X

    long_name :

    grid index in x for variables at tracer and 'v' locations

    swap_dim :

    XC

    comment :

    In the Arakawa C-grid system, tracer (e.g., THETA) and 'v' variables (e.g., VVEL) have the same x coordinate on the model grid.

    coverage_content_type :

    coordinate

    origname :

    i

    fullnamepath :

    /i

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype=int32)

  • j
    (j)
    int32
    0 1 2 3 4 5 6 ... 84 85 86 87 88 89

    axis :

    Y

    long_name :

    grid index in y for variables at tracer and 'u' locations

    swap_dim :

    YC

    comment :

    In the Arakawa C-grid system, tracer (e.g., THETA) and 'u' variables (e.g., UVEL) have the same y coordinate on the model grid.

    coverage_content_type :

    coordinate

    origname :

    j

    fullnamepath :

    /j

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype=int32)

  • k
    (k)
    int32
    0 1 2 3 4 5 6 ... 44 45 46 47 48 49

    axis :

    Z

    long_name :

    grid index in z for tracer variables

    swap_dim :

    Z

    coverage_content_type :

    coordinate

    origname :

    k

    fullnamepath :

    /k

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49], dtype=int32)

  • tile
    (tile)
    int32
    0 1 2 3 4 5 6 7 8 9 10 11 12

    long_name :

    lat-lon-cap tile index

    comment :

    The ECCO V4 horizontal model grid is divided into 13 tiles of 90x90 cells for convenience.

    coverage_content_type :

    coordinate

    origname :

    tile

    fullnamepath :

    /tile

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12], dtype=int32)

  • time
    (time)
    datetime64[ns]
    2017-01-16T12:00:00

    long_name :

    center time of averaging period

    axis :

    T

    bounds :

    time_bnds

    coverage_content_type :

    coordinate

    standard_name :

    time

    origname :

    time

    fullnamepath :

    /time

    Maps :

    ()

    array(['2017-01-16T12:00:00.000000000'], dtype='datetime64[ns]')

• Data variables: (1)
  • THETA
    (time, k, tile, j, i)
    float32
    ...

    long_name :

    Potential temperature

    units :

    degree_C

    coverage_content_type :

    modelResult

    standard_name :

    sea_water_potential_temperature

    comment :

    Sea water potential temperature is the temperature a parcel of sea water would have if moved adiabatically to sea level pressure. Note: the equation of state is a modified UNESCO formula by Jackett and McDougall (1995), which uses the model variable potential temperature as input assuming a horizontally and temporally constant pressure of $p_0=-g ho_{0} z$.

    valid_min :

    -2.2909388542175293

    valid_max :

    36.032955169677734

    origname :

    THETA

    fullnamepath :

    /THETA

    Maps :

    ()

    [5265000 values with dtype=float32]

• Indexes: (5)
  • i
    PandasIndex

    PandasIndex(Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype='int32', name='i'))

  • j
    PandasIndex

    PandasIndex(Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype='int32', name='j'))

  • k
    PandasIndex

    PandasIndex(Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
          dtype='int32', name='k'))

  • tile
    PandasIndex

    PandasIndex(Index([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], dtype='int32', name='tile'))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2017-01-16 12:00:00'], dtype='datetime64[ns]', name='time', freq=None))

• Attributes: (62)

  acknowledgement :

  This research was carried out by the Jet Propulsion Laboratory, managed by the California Institute of Technology under a contract with the National Aeronautics and Space Administration.

  author :

  Ian Fenty and Ou Wang

  cdm_data_type :

  Grid

  comment :

  Fields provided on the curvilinear lat-lon-cap 90 (llc90) native grid used in the ECCO model.

  Conventions :

  CF-1.8, ACDD-1.3

  coordinates_comment :

  Note: the global 'coordinates' attribute describes auxillary coordinates.

  creator_email :

  ecco-group@mit.edu

  creator_institution :

  NASA Jet Propulsion Laboratory (JPL)

  creator_name :

  ECCO Consortium

  creator_type :

  group

  creator_url :

  https://ecco-group.org

  date_created :

  2020-12-18T14:39:59

  date_issued :

  2020-12-18T14:39:59

  date_metadata_modified :

  2021-03-15T21:56:21

  date_modified :

  2021-03-15T21:56:21

  geospatial_bounds_crs :

  EPSG:4326

  geospatial_lat_max :

  90.0

  geospatial_lat_min :

  -90.0

  geospatial_lat_resolution :

  variable

  geospatial_lat_units :

  degrees_north

  geospatial_lon_max :

  180.0

  geospatial_lon_min :

  -180.0

  geospatial_lon_resolution :

  variable

  geospatial_lon_units :

  degrees_east

  geospatial_vertical_max :

  0.0

  geospatial_vertical_min :

  -6134.5

  geospatial_vertical_positive :

  up

  geospatial_vertical_resolution :

  variable

  geospatial_vertical_units :

  meter

  history :

  Inaugural release of an ECCO Central Estimate solution to PO.DAAC

  id :

  10.5067/ECL5M-OTS44

  institution :

  NASA Jet Propulsion Laboratory (JPL)

  instrument_vocabulary :

  GCMD instrument keywords

  keywords :

  EARTH SCIENCE > OCEANS > SALINITY/DENSITY > SALINITY, EARTH SCIENCE SERVICES > MODELS > EARTH SCIENCE REANALYSES/ASSIMILATION MODELS, EARTH SCIENCE > OCEANS > OCEAN TEMPERATURE > POTENTIAL TEMPERATURE

  keywords_vocabulary :

  NASA Global Change Master Directory (GCMD) Science Keywords

  license :

  Public Domain

  metadata_link :

  https://cmr.earthdata.nasa.gov/search/collections.umm_json?ShortName=ECCO_L4_TEMP_SALINITY_LLC0090GRID_MONTHLY_V4R4

  naming_authority :

  gov.nasa.jpl

  platform :

  ERS-1/2, TOPEX/Poseidon, Geosat Follow-On (GFO), ENVISAT, Jason-1, Jason-2, CryoSat-2, SARAL/AltiKa, Jason-3, AVHRR, Aquarius, SSM/I, SSMIS, GRACE, DTU17MDT, Argo, WOCE, GO-SHIP, MEOP, Ice Tethered Profilers (ITP)

  platform_vocabulary :

  GCMD platform keywords

  processing_level :

  L4

  product_name :

  OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.nc

  product_time_coverage_end :

  2018-01-01T00:00:00

  product_time_coverage_start :

  1992-01-01T12:00:00

  product_version :

  Version 4, Release 4

  program :

  NASA Physical Oceanography, Cryosphere, Modeling, Analysis, and Prediction (MAP)

  project :

  Estimating the Circulation and Climate of the Ocean (ECCO)

  publisher_email :

  podaac@podaac.jpl.nasa.gov

  publisher_institution :

  PO.DAAC

  publisher_name :

  Physical Oceanography Distributed Active Archive Center (PO.DAAC)

  publisher_type :

  institution

  publisher_url :

  https://podaac.jpl.nasa.gov

  references :

  ECCO Consortium, Fukumori, I., Wang, O., Fenty, I., Forget, G., Heimbach, P., & Ponte, R. M. 2020. Synopsis of the ECCO Central Production Global Ocean and Sea-Ice State Estimate (Version 4 Release 4). doi:10.5281/zenodo.3765928

  source :

  The ECCO V4r4 state estimate was produced by fitting a free-running solution of the MITgcm (checkpoint 66g) to satellite and in situ observational data in a least squares sense using the adjoint method

  standard_name_vocabulary :

  NetCDF Climate and Forecast (CF) Metadata Convention

  summary :

  This dataset provides monthly-averaged ocean potential temperature and salinity on the lat-lon-cap 90 (llc90) native model grid from the ECCO Version 4 Release 4 (V4r4) ocean and sea-ice state estimate. Estimating the Circulation and Climate of the Ocean (ECCO) state estimates are dynamically and kinematically-consistent reconstructions of the three-dimensional, time-evolving ocean, sea-ice, and surface atmospheric states. ECCO V4r4 is a free-running solution of a global, nominally 1-degree configuration of the MIT general circulation model (MITgcm) that has been fit to observations in a least-squares sense. Observational data constraints used in V4r4 include sea surface height (SSH) from satellite altimeters [ERS-1/2, TOPEX/Poseidon, GFO, ENVISAT, Jason-1,2,3, CryoSat-2, and SARAL/AltiKa]; sea surface temperature (SST) from satellite radiometers [AVHRR], sea surface salinity (SSS) from the Aquarius satellite radiometer/scatterometer, ocean bottom pressure (OBP) from the GRACE satellite gravimeter; sea-ice concentration from satellite radiometers [SSM/I and SSMIS], and in-situ ocean temperature and salinity measured with conductivity-temperature-depth (CTD) sensors and expendable bathythermographs (XBTs) from several programs [e.g., WOCE, GO-SHIP, Argo, and others] and platforms [e.g., research vessels, gliders, moorings, ice-tethered profilers, and instrumented pinnipeds]. V4r4 covers the period 1992-01-01T12:00:00 to 2018-01-01T00:00:00.

  time_coverage_duration :

  P1M

  time_coverage_end :

  2017-02-01T00:00:00

  time_coverage_resolution :

  P1M

  time_coverage_start :

  2017-01-01T00:00:00

  title :

  ECCO Ocean Temperature and Salinity - Monthly Mean llc90 Grid (Version 4 Release 4)

  uuid :

  f5b7028c-4181-11eb-b7e6-0cc47a3f47b1

Multiple files in parallel

We can exploit xarray’s intuitive concat + merge capabilities to read multiple data URL in parallel. To accomplish that, we need to create a list of all relevant URLs that each point to an OPeNDAP dataset.

Tip

In this case we will use a session that caches the response. It can handle the authentication too!

Temp_2017 = [baseURL + Temp_Salt + year + f'{i:02}' + end_ + CE for i in range(1, 13)]

Temp_2017[:4]

['dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090?dap4.ce=/THETA;/time;/k;/tile;/j;/i',
 'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-02_ECCO_V4r4_native_llc0090?dap4.ce=/THETA;/time;/k;/tile;/j;/i',
 'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-03_ECCO_V4r4_native_llc0090?dap4.ce=/THETA;/time;/k;/tile;/j;/i',
 'dap4://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-04_ECCO_V4r4_native_llc0090?dap4.ce=/THETA;/time;/k;/tile;/j;/i']

cache_session = create_session(use_cache=True) # caching the session
cache_session.cache.clear() # clear all cache to demonstrate the behavior. You do not have to do this!

Tip

You can use the consolidated_metadata method to rapidly download all metadata associated with a datacube, and cache it. This is a feature of pydap >= 3.5.5.

from pydap.client import consolidate_metadata

%%time
consolidate_metadata(Temp_2017, cache_session)

datacube has dimensions {'i[0:1:89]', 'j[0:1:89]', 'k[0:1:49]', 'time[0:1:0]', 'tile[0:1:12]'}

CPU times: user 619 ms, sys: 354 ms, total: 973 ms
Wall time: 11.4 s

cache_session.cache.urls()

['https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.dap?dap4.ce=i%5B0%3A1%3A89%5D',
 'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.dap?dap4.ce=j%5B0%3A1%3A89%5D',
 'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.dap?dap4.ce=k%5B0%3A1%3A49%5D',
 'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.dap?dap4.ce=tile%5B0%3A1%3A12%5D',
 'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-01_ECCO_V4r4_native_llc0090.dap?dap4.ce=time%5B0%3A1%3A0%5D',
 'https://opendap.earthdata.nasa.gov/providers/POCLOUD/collections/ECCO%20Ocean%20Temperature%20and%20Salinity%20-%20Monthly%20Mean%20llc90%20Grid%20(Version%204%20Release%204)/granules/OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-10_ECCO_V4r4_native_llc0090.dmr?dap4.ce=%2FTHETA%3B%2Ftime%3B%2Fk%3B%2Ftile%3B%2Fj%3B%2Fi']

%%time
theta_salt_ds = xr.open_mfdataset(
    Temp_2017, 
    engine='pydap',
    session=cache_session, 
    parallel=True,
    combine='nested',
    concat_dim='time',
    decode_times=False
)

CPU times: user 242 ms, sys: 120 ms, total: 363 ms
Wall time: 300 ms

theta_salt_ds

<xarray.Dataset> Size: 253MB
Dimensions:  (time: 12, k: 50, tile: 13, j: 90, i: 90)
Coordinates:
  * i        (i) int32 360B 0 1 2 3 4 5 6 7 8 9 ... 81 82 83 84 85 86 87 88 89
  * j        (j) int32 360B 0 1 2 3 4 5 6 7 8 9 ... 81 82 83 84 85 86 87 88 89
  * k        (k) int32 200B 0 1 2 3 4 5 6 7 8 9 ... 41 42 43 44 45 46 47 48 49
  * tile     (tile) int32 52B 0 1 2 3 4 5 6 7 8 9 10 11 12
  * time     (time) int32 48B 219528 219528 219528 ... 219528 219528 219528
Data variables:
    THETA    (time, k, tile, j, i) float32 253MB dask.array<chunksize=(1, 50, 13, 90, 90), meta=np.ndarray>
Attributes: (12/62)
    acknowledgement:                 This research was carried out by the Jet...
    author:                          Ian Fenty and Ou Wang
    cdm_data_type:                   Grid
    comment:                         Fields provided on the curvilinear lat-l...
    Conventions:                     CF-1.8, ACDD-1.3
    coordinates_comment:             Note: the global 'coordinates' attribute...
    ...                              ...
    time_coverage_duration:          P1M
    time_coverage_end:               2017-11-01T00:00:00
    time_coverage_resolution:        P1M
    time_coverage_start:             2017-10-01T00:00:00
    title:                           ECCO Ocean Temperature and Salinity - Mo...
    uuid:                            fba8c270-4181-11eb-a317-0cc47a3f47df

xarray.Dataset

• Dimensions:
  • time: 12
  • k: 50
  • tile: 13
  • j: 90
  • i: 90
• Coordinates: (5)
  • i
    (i)
    int32
    0 1 2 3 4 5 6 ... 84 85 86 87 88 89

    axis :

    X

    long_name :

    grid index in x for variables at tracer and 'v' locations

    swap_dim :

    XC

    comment :

    In the Arakawa C-grid system, tracer (e.g., THETA) and 'v' variables (e.g., VVEL) have the same x coordinate on the model grid.

    coverage_content_type :

    coordinate

    origname :

    i

    fullnamepath :

    /i

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype=int32)

  • j
    (j)
    int32
    0 1 2 3 4 5 6 ... 84 85 86 87 88 89

    axis :

    Y

    long_name :

    grid index in y for variables at tracer and 'u' locations

    swap_dim :

    YC

    comment :

    In the Arakawa C-grid system, tracer (e.g., THETA) and 'u' variables (e.g., UVEL) have the same y coordinate on the model grid.

    coverage_content_type :

    coordinate

    origname :

    j

    fullnamepath :

    /j

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype=int32)

  • k
    (k)
    int32
    0 1 2 3 4 5 6 ... 44 45 46 47 48 49

    axis :

    Z

    long_name :

    grid index in z for tracer variables

    swap_dim :

    Z

    coverage_content_type :

    coordinate

    origname :

    k

    fullnamepath :

    /k

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49], dtype=int32)

  • tile
    (tile)
    int32
    0 1 2 3 4 5 6 7 8 9 10 11 12

    long_name :

    lat-lon-cap tile index

    comment :

    The ECCO V4 horizontal model grid is divided into 13 tiles of 90x90 cells for convenience.

    coverage_content_type :

    coordinate

    origname :

    tile

    fullnamepath :

    /tile

    Maps :

    ()

    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12], dtype=int32)

  • time
    (time)
    int32
    219528 219528 ... 219528 219528

    long_name :

    center time of averaging period

    axis :

    T

    bounds :

    time_bnds

    coverage_content_type :

    coordinate

    standard_name :

    time

    units :

    hours since 1992-01-01T12:00:00

    calendar :

    proleptic_gregorian

    origname :

    time

    fullnamepath :

    /time

    Maps :

    ()

    array([219528, 219528, 219528, 219528, 219528, 219528, 219528, 219528, 219528,
           219528, 219528, 219528], dtype=int32)

• Data variables: (1)
  • THETA
    (time, k, tile, j, i)
    float32
    dask.array<chunksize=(1, 50, 13, 90, 90), meta=np.ndarray>

    long_name :

    Potential temperature

    units :

    degree_C

    coverage_content_type :

    modelResult

    standard_name :

    sea_water_potential_temperature

    comment :

    Sea water potential temperature is the temperature a parcel of sea water would have if moved adiabatically to sea level pressure. Note: the equation of state is a modified UNESCO formula by Jackett and McDougall (1995), which uses the model variable potential temperature as input assuming a horizontally and temporally constant pressure of $p_0=-g ho_{0} z$.

    valid_min :

    -2.2909388542175293

    valid_max :

    36.032955169677734

    origname :

    THETA

    fullnamepath :

    /THETA

    Maps :

    ()

    Array Chunk Bytes 241.01 MiB 20.08 MiB Shape (12, 50, 13, 90, 90) (1, 50, 13, 90, 90) Dask graph 12 chunks in 25 graph layers Data type float32 numpy.ndarray

• Indexes: (5)
  • i
    PandasIndex

    PandasIndex(Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype='int32', name='i'))

  • j
    PandasIndex

    PandasIndex(Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53,
           54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71,
           72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89],
          dtype='int32', name='j'))

  • k
    PandasIndex

    PandasIndex(Index([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
           18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35,
           36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
          dtype='int32', name='k'))

  • tile
    PandasIndex

    PandasIndex(Index([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], dtype='int32', name='tile'))

  • time
    PandasIndex

    PandasIndex(Index([219528, 219528, 219528, 219528, 219528, 219528, 219528, 219528, 219528,
           219528, 219528, 219528],
          dtype='int32', name='time'))

• Attributes: (62)

  acknowledgement :

  This research was carried out by the Jet Propulsion Laboratory, managed by the California Institute of Technology under a contract with the National Aeronautics and Space Administration.

  author :

  Ian Fenty and Ou Wang

  cdm_data_type :

  Grid

  comment :

  Fields provided on the curvilinear lat-lon-cap 90 (llc90) native grid used in the ECCO model.

  Conventions :

  CF-1.8, ACDD-1.3

  coordinates_comment :

  Note: the global 'coordinates' attribute describes auxillary coordinates.

  creator_email :

  ecco-group@mit.edu

  creator_institution :

  NASA Jet Propulsion Laboratory (JPL)

  creator_name :

  ECCO Consortium

  creator_type :

  group

  creator_url :

  https://ecco-group.org

  date_created :

  2020-12-18T14:40:09

  date_issued :

  2020-12-18T14:40:09

  date_metadata_modified :

  2021-03-15T21:56:26

  date_modified :

  2021-03-15T21:56:26

  geospatial_bounds_crs :

  EPSG:4326

  geospatial_lat_max :

  90.0

  geospatial_lat_min :

  -90.0

  geospatial_lat_resolution :

  variable

  geospatial_lat_units :

  degrees_north

  geospatial_lon_max :

  180.0

  geospatial_lon_min :

  -180.0

  geospatial_lon_resolution :

  variable

  geospatial_lon_units :

  degrees_east

  geospatial_vertical_max :

  0.0

  geospatial_vertical_min :

  -6134.5

  geospatial_vertical_positive :

  up

  geospatial_vertical_resolution :

  variable

  geospatial_vertical_units :

  meter

  history :

  Inaugural release of an ECCO Central Estimate solution to PO.DAAC

  id :

  10.5067/ECL5M-OTS44

  institution :

  NASA Jet Propulsion Laboratory (JPL)

  instrument_vocabulary :

  GCMD instrument keywords

  keywords :

  EARTH SCIENCE > OCEANS > OCEAN TEMPERATURE > POTENTIAL TEMPERATURE, EARTH SCIENCE SERVICES > MODELS > EARTH SCIENCE REANALYSES/ASSIMILATION MODELS, EARTH SCIENCE > OCEANS > SALINITY/DENSITY > SALINITY

  keywords_vocabulary :

  NASA Global Change Master Directory (GCMD) Science Keywords

  license :

  Public Domain

  metadata_link :

  https://cmr.earthdata.nasa.gov/search/collections.umm_json?ShortName=ECCO_L4_TEMP_SALINITY_LLC0090GRID_MONTHLY_V4R4

  naming_authority :

  gov.nasa.jpl

  platform :

  ERS-1/2, TOPEX/Poseidon, Geosat Follow-On (GFO), ENVISAT, Jason-1, Jason-2, CryoSat-2, SARAL/AltiKa, Jason-3, AVHRR, Aquarius, SSM/I, SSMIS, GRACE, DTU17MDT, Argo, WOCE, GO-SHIP, MEOP, Ice Tethered Profilers (ITP)

  platform_vocabulary :

  GCMD platform keywords

  processing_level :

  L4

  product_name :

  OCEAN_TEMPERATURE_SALINITY_mon_mean_2017-10_ECCO_V4r4_native_llc0090.nc

  product_time_coverage_end :

  2018-01-01T00:00:00

  product_time_coverage_start :

  1992-01-01T12:00:00

  product_version :

  Version 4, Release 4

  program :

  NASA Physical Oceanography, Cryosphere, Modeling, Analysis, and Prediction (MAP)

  project :

  Estimating the Circulation and Climate of the Ocean (ECCO)

  publisher_email :

  podaac@podaac.jpl.nasa.gov

  publisher_institution :

  PO.DAAC

  publisher_name :

  Physical Oceanography Distributed Active Archive Center (PO.DAAC)

  publisher_type :

  institution

  publisher_url :

  https://podaac.jpl.nasa.gov

  references :

  ECCO Consortium, Fukumori, I., Wang, O., Fenty, I., Forget, G., Heimbach, P., & Ponte, R. M. 2020. Synopsis of the ECCO Central Production Global Ocean and Sea-Ice State Estimate (Version 4 Release 4). doi:10.5281/zenodo.3765928

  source :

  The ECCO V4r4 state estimate was produced by fitting a free-running solution of the MITgcm (checkpoint 66g) to satellite and in situ observational data in a least squares sense using the adjoint method

  standard_name_vocabulary :

  NetCDF Climate and Forecast (CF) Metadata Convention

  summary :

  This dataset provides monthly-averaged ocean potential temperature and salinity on the lat-lon-cap 90 (llc90) native model grid from the ECCO Version 4 Release 4 (V4r4) ocean and sea-ice state estimate. Estimating the Circulation and Climate of the Ocean (ECCO) state estimates are dynamically and kinematically-consistent reconstructions of the three-dimensional, time-evolving ocean, sea-ice, and surface atmospheric states. ECCO V4r4 is a free-running solution of a global, nominally 1-degree configuration of the MIT general circulation model (MITgcm) that has been fit to observations in a least-squares sense. Observational data constraints used in V4r4 include sea surface height (SSH) from satellite altimeters [ERS-1/2, TOPEX/Poseidon, GFO, ENVISAT, Jason-1,2,3, CryoSat-2, and SARAL/AltiKa]; sea surface temperature (SST) from satellite radiometers [AVHRR], sea surface salinity (SSS) from the Aquarius satellite radiometer/scatterometer, ocean bottom pressure (OBP) from the GRACE satellite gravimeter; sea-ice concentration from satellite radiometers [SSM/I and SSMIS], and in-situ ocean temperature and salinity measured with conductivity-temperature-depth (CTD) sensors and expendable bathythermographs (XBTs) from several programs [e.g., WOCE, GO-SHIP, Argo, and others] and platforms [e.g., research vessels, gliders, moorings, ice-tethered profilers, and instrumented pinnipeds]. V4r4 covers the period 1992-01-01T12:00:00 to 2018-01-01T00:00:00.

  time_coverage_duration :

  P1M

  time_coverage_end :

  2017-11-01T00:00:00

  time_coverage_resolution :

  P1M

  time_coverage_start :

  2017-10-01T00:00:00

  title :

  ECCO Ocean Temperature and Salinity - Monthly Mean llc90 Grid (Version 4 Release 4)

  uuid :

  fba8c270-4181-11eb-a317-0cc47a3f47df

Finally, we plot THETA

Variable = [theta_salt_ds['THETA'][0, 0, i, :, :] for i in range(13)]
clevels = np.linspace(-5, 30, 100)
cMap='RdBu_r'

fig, axes = plt.subplots(nrows=4, ncols=4, figsize=(8, 8), gridspec_kw={'hspace':0.01, 'wspace':0.01})
AXES_NR = [
    axes[3, 0], axes[2, 0], axes[1, 0], axes[3, 1], axes[2, 1], axes[1, 1],
]
AXES_CAP = [axes[0, 0]]
AXES_R = [
    axes[1, 2], axes[2, 2], axes[3, 2], axes[1, 3], axes[2, 3], axes[3, 3],
]
for i in range(len(AXES_NR)):
    AXES_NR[i].contourf(Variable[i], clevels, cmap=cMap)

for i in range(len(AXES_CAP)):
    AXES_CAP[i].contourf(Variable[6].transpose()[:, ::-1], clevels, cmap=cMap)

for i in range(len(AXES_R)):
    AXES_R[i].contourf(Variable[7+i].transpose()[::-1, :], clevels, cmap=cMap)

for ax in np.ravel(axes):
    ax.axis('off')
    plt.setp(ax.get_xticklabels(), visible=False)
    plt.setp(ax.get_yticklabels(), visible=False)

plt.show()

Fig. 3. Surface temperature, plotted on a horizontal layout that approximates lat-lon display. Data on the arctic cap, however, remains on a polar coordinate grid.