Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 36 additions & 5 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ The goal of datamatch is to pull environmental data from Copernicus Marine Servi
- [Set up](#set-up) — the Copernicus client, sign-in, and where downloads are cached
- [Quick start](#quick-start)
- [One call per product](#one-call-per-product) — why `SST` and `CHL` are two fetches
- [Monthly or daily](#monthly-or-daily) — monthly by default, and how to fetch particular dates
- [Monthly or daily](#monthly-or-daily) — monthly by default; every day of a period, or particular dates
- [Every day in a period](#every-day-in-a-period)
- [Particular days](#particular-days)
- [Downloads run in parallel](#downloads-run-in-parallel) — `n_workers`, and what is already cached

**Choosing what to fetch**
Expand Down Expand Up @@ -162,21 +164,37 @@ matches to the nearest cell whatever its size.
### Monthly or daily

**Fetches are monthly means by default**, and a call that mentions neither
`frequency` nor `dates` behaves exactly as it always has. `frequency = "daily"`
uses the daily datasets instead, expanding each requested month into its days:
`frequency` nor `dates` behaves exactly as it always has.

There are two ways to ask for daily data, for two different jobs:

| Want | Use | Gives |
|---|---|---|
| Every day in a period | `frequency = "daily"` with `years` and `months` | a continuous series |
| Particular days | `dates` | only those dates |

#### Every day in a period

`frequency = "daily"` uses the daily datasets, expanding each requested month
into all of its days:

```{r quickstart-daily, eval = FALSE}
sst <- accessEnvDat(vars = c("SST", "MLD"), frequency = "daily",
years = 2015, months = 4:6, bounding_box = bb)
```

That is the form to use for a continuous series — a time series at one station,
an animation, anything where the gaps between days would matter.

Daily is a real cost, not a flag: three months is 91 downloads rather than 3,
and 91 grids rather than 3 in memory. A decade of daily data over a large box
will not fit in a laptop's RAM as an `sf` object, and is better fetched a season
at a time.

`dates` is the other way to keep that in hand. It names the exact dates to
fetch, so only the days that matter are downloaded:
#### Particular days

`dates` names the exact dates to fetch, so only the days that matter are
downloaded:

```{r quickstart-dates, eval = FALSE}
accessEnvDat(vars = "SST", dates = c("20150402", "20150517"), bounding_box = bb)
Expand All @@ -201,6 +219,19 @@ an error naming it, not a silently dropped request.
`dates` says which time steps to fetch, so `years` and `months` are neither
needed nor accepted alongside it, and it implies `frequency = "daily"`.

It is also how a long record is thinned rather than fetched whole, since any
sequence of dates will do:

```{r quickstart-dates-thin, eval = FALSE}
# Weekly through a decade: 574 downloads rather than 4,017
accessEnvDat(vars = "SST", bounding_box = bb,
dates = seq(as.Date("2005-01-01"), as.Date("2015-12-31"), by = "week"))

# Or the same day each month, if that is genuinely what you want
accessEnvDat(vars = "SST", bounding_box = bb,
dates = seq(as.Date("2005-01-15"), as.Date("2015-12-15"), by = "month"))
```

Fetching dates is not the same as averaging a month. Three dates are a sample of
the month, carrying whatever weather fell on them; a monthly mean is the month.
Which you want depends on whether the observations you are matching are
Expand Down
43 changes: 38 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ The goal of datamatch is to pull environmental data from Copernicus Marine Servi
- [Set up](#set-up) — the Copernicus client, sign-in, and where downloads are cached
- [Quick start](#quick-start)
- [One call per product](#one-call-per-product) — why `SST` and `CHL` are two fetches
- [Monthly or daily](#monthly-or-daily) — monthly by default, and how to fetch particular dates
- [Monthly or daily](#monthly-or-daily) — monthly by default; every day of a period, or particular dates
- [Every day in a period](#every-day-in-a-period)
- [Particular days](#particular-days)
- [Downloads run in parallel](#downloads-run-in-parallel) — `n_workers`, and what is already cached

**Choosing what to fetch**
Expand Down Expand Up @@ -158,22 +160,38 @@ matches to the nearest cell whatever its size.
### Monthly or daily

**Fetches are monthly means by default**, and a call that mentions neither
`frequency` nor `dates` behaves exactly as it always has. `frequency = "daily"`
uses the daily datasets instead, expanding each requested month into its days:
`frequency` nor `dates` behaves exactly as it always has.

There are two ways to ask for daily data, for two different jobs:

| Want | Use | Gives |
|---|---|---|
| Every day in a period | `frequency = "daily"` with `years` and `months` | a continuous series |
| Particular days | `dates` | only those dates |

#### Every day in a period

`frequency = "daily"` uses the daily datasets, expanding each requested month
into all of its days:


``` r
sst <- accessEnvDat(vars = c("SST", "MLD"), frequency = "daily",
years = 2015, months = 4:6, bounding_box = bb)
```

That is the form to use for a continuous series — a time series at one station,
an animation, anything where the gaps between days would matter.

Daily is a real cost, not a flag: three months is 91 downloads rather than 3,
and 91 grids rather than 3 in memory. A decade of daily data over a large box
will not fit in a laptop's RAM as an `sf` object, and is better fetched a season
at a time.

`dates` is the other way to keep that in hand. It names the exact dates to
fetch, so only the days that matter are downloaded:
#### Particular days

`dates` names the exact dates to fetch, so only the days that matter are
downloaded:


``` r
Expand All @@ -200,6 +218,20 @@ an error naming it, not a silently dropped request.
`dates` says which time steps to fetch, so `years` and `months` are neither
needed nor accepted alongside it, and it implies `frequency = "daily"`.

It is also how a long record is thinned rather than fetched whole, since any
sequence of dates will do:


``` r
# Weekly through a decade: 574 downloads rather than 4,017
accessEnvDat(vars = "SST", bounding_box = bb,
dates = seq(as.Date("2005-01-01"), as.Date("2015-12-31"), by = "week"))

# Or the same day each month, if that is genuinely what you want
accessEnvDat(vars = "SST", bounding_box = bb,
dates = seq(as.Date("2005-01-15"), as.Date("2015-12-15"), by = "month"))
```

Fetching dates is not the same as averaging a month. Three dates are a sample of
the month, carrying whatever weather fell on them; a monthly mean is the month.
Which you want depends on whether the observations you are matching are
Expand Down Expand Up @@ -813,6 +845,7 @@ Four of the six indices are still growing. Downloads are cached, and the cache
expires on an interval matched to how often each provider actually publishes, so
a living index re-downloads on its own without being asked:


``` r
climate_index_status() # what is cached, how old, what is due
refresh_climate_index() # force a re-fetch of everything still growing
Expand Down
Loading