Skip to content

GRAFS plot dataset#63

Open
AliceBeckmann wants to merge 159 commits intomainfrom
alice/grafs_plot
Open

GRAFS plot dataset#63
AliceBeckmann wants to merge 159 commits intomainfrom
alice/grafs_plot

Conversation

@AliceBeckmann
Copy link
Collaborator

@AliceBeckmann AliceBeckmann commented Oct 27, 2025

TODOs:

  • Changing the <CRP_LS> label from emissions to losses in crop production (the part that was lost and not used).

GRAFS plot dataset description
This PR introduces the main function create_grafs_plot_df(), which includes helper functions to generate a dataset of nitrogen (MgN) flows by province and year as well as land use information in Spain. The data are assigned to labels, defined by Alfredo Rodriguez, which are needed to create GRAFS plots with Alfredo’s R package.

The resulting tibble is designed to feed directly into the GRAFS visualization R package.


Label description

Land use and land N production
Land use areas and associated N production are taken from n_balance_ygpit_all.
They are aggregated by province and year and mapped to labels such as:
cropland N and area ({ARAiN}, {ARArN}, {ARAiha}, {ARArha}),
permanent crops ({PERiN}, {PERrN}),
horticulture ({HORiN}, {HORrN}),
forest and semi-natural systems ({FORN}),
grassland ({HAGRASS}, {GRASSMha})
greenhouse systems ({GREHN}).
Both irrigated and rainfed systems are treated separately.

Crop and grass N consumption
N flows (consumption) from cropland and grassland are taken from create_n_prov_destiny() and tranformed to labels including:
crops to population ({CROPS_TO_POP}),
crops to livestock ({CROPS_TO_LIVESTOCK}),
grass to livestock ({GRASS_TO_LIVESTOCK}),
crop exports ({CROP_EXPORT}),
crop N used for other uses ({CRP_OTHUSES}).

Livestock production
Livestock N production is taken from create_n_prov_destiny() and livestock_prod_ygps.
It is separated into edible and non-edible products and mapped to:
edible livestock products ({LV_EDBL}),
non-edible livestock products ({LVSTCK_NOEDIBLE}),
milk and dairy products ({LVST_MILK}),
exported livestock products ({LIVESTOCK_EXPORTED}).

Livestock losses and totals
Livestock N losses are taken from n_excretion_ygs and population-related other uses.
They are represented using the labels:
livestock losses ({AN_LS}),
livestock other uses ({AN_OTH}),
total livestock losses ({AN_LS_OTH}),
total livestock N ({LVSTCKTOTN}),
gaseous N losses ({LIVGASLOSS}).

Feed and internal transfers
Feed flows from cropland to livestock are taken from create_n_prov_destiny() and split into:
ruminant feed ({RCRTOLVSTCK_R}),
monogastric feed ({MCRTOLVSTCK_M}).

Imports and N soil inputs
N imports and N soil inputs are taken from create_n_prov_destiny() and n_balance_ygpit_all.
They are mapped to labels such as:
crop and animal product imports ({CROP_POPIMPORT}, {IMANOT}, {IMPORT_ANIMALCR}),
synthetic fertilizers ({SYNTHF}, {SYNTHF_TOTAL}),
biological fixation and atmospheric deposition ({FIXCR}, {OXDEPCROPS}, {FIX_DEP_CR}, {FIX_DEP_GRASS}),
wastewater inputs from population ({WASTEWATER}).

Livestock units and population
Livestock units (LU) are calculated from livestock_prod_ygps stock numbers and included using:
ruminant LU ({RUMIANTSLU}, {RUMIANTSMLU}),
monogastric LU ({MONOGLU}, {MONOGMLU}).
Population data are taken from population_yg and included as {POPULATIONM}.

Final steps
All datasets are merged into a single standardized table.
Missing labels required for plotting are added with zero values.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

air

[air] reported by reviewdog 🐶

print(paste0(png_out," created!"))


[air] reported by reviewdog 🐶


[air] reported by reviewdog 🐶

year_info <- function(YEARS){


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 340 to 345 in 308f210

for(i in 2:length(YEARS)){
if(YEARS[i]!=(YEARS[i-1]+1)){
label <- paste0(label,"-",YEARS[i-1],"_",YEARS[i])
}else{
if(i==length(YEARS)){
label <- paste0(label,"-",YEARS[i])


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 349 to 350 in 308f210

for(i in 1:length(unique(YEARS))){
label <- str_replace_all(label,paste0(YEARS[i],"-",YEARS[i]),paste0(YEARS[i],""))


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 355 to 372 in 308f210

create_GRAFS <- function(XLSX_INPUTS,
PATH_OUTPUTS,
XML_BASE,
TABLA_ID_XML,
DRAW_IO_EXE,
REGIONS,
PERIODS,
DECIMALES_XML=0,
MAX_WIDTH_ARROWS=25,
VAL_MAX_WIDTH=1000,
INCREASE_COLOR="#97cde5",
DECREASE_COLOR="#a9d77f",
OVERWRITE=TRUE,
VERBOSE=FALSE,
UNITch=NA,
LABELch=NA,
DATOch=NA){


[air] reported by reviewdog 🐶


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 376 to 381 in 308f210

dir.create(PATH_OUTPUTS,showWarnings = F)
dir.create(paste0(PATH_OUTPUTS,"xml/"),showWarnings = F)
dir.create(paste0(PATH_OUTPUTS,"png/"),showWarnings = F)
XML_temp <- str_replace_all(XML_BASE,".xml","_temp.xml")


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 386 to 387 in 308f210

if(length(unique(d$year))==1){
if(unique(d$year)==9999){


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 390 to 391 in 308f210

d <- rbind(d,d2)
}}


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 394 to 397 in 308f210

d <- subset(d,!grepl("WIDTH_MAX",label))
for(PERIOD in 1:length(PERIODS)){


[air] reported by reviewdog 🐶

tPER <- PERIODS[PERIOD]


[air] reported by reviewdog 🐶

if(grepl("-",tPER)){


[air] reported by reviewdog 🐶

aux <- str_split(tPER,"-")


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 404 to 408 in 308f210

auxt <- paste0("YEARS <- ",aux[1])
eval(parse(text=auxt))
auxt <- paste0("YEARS_CHANGE <- ",aux[2])
eval(parse(text=auxt))
}else{


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 410 to 411 in 308f210

auxt <- paste0("YEARS <- ",tPER)
eval(parse(text=auxt))


[air] reported by reviewdog 🐶

for(PROV_ACT in REGIONS){


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 416 to 419 in 308f210

dact <- subset(d,province==PROV_ACT & is.element(year,YEARS))
if(PLOT_CHANGE){
dactch <- subset(d,province==PROV_ACT & is.element(year,YEARS_CHANGE))


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 422 to 425 in 308f210

FACT_XML <- paste0(PATH_OUTPUTS,"xml/","GRAFS_",PROV_ACT,"_P",PERIOD,"_MEAN_",year_info(YEARS),".xml")
if(file.exists(FACT_XML) & !OVERWRITE){


[air] reported by reviewdog 🐶


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 432 to 437 in 308f210

if(PLOT_CHANGE){
txml <- change(txml,"<YEARCHANGE>",paste0("(changes_",YEARS_CHANGE[1],"-",max(YEARS_CHANGE),")"),T_ID_ARROW=T_ID_ARROW,VAL_MAX_WIDTH=VAL_MAX_WIDTH,DECIMALES_XML=DECIMALES_XML,XML_temp=XML_temp,MAX_WIDTH_ARROWS=MAX_WIDTH_ARROWS,PLOT_CHANGE=PLOT_CHANGE,DATOch=DATOch,
INCREASE_COLOR=INCREASE_COLOR,DECREASE_COLOR=DECREASE_COLOR)
}else{
txml <- change(txml,"<YEARCHANGE>","",T_ID_ARROW=T_ID_ARROW,VAL_MAX_WIDTH=VAL_MAX_WIDTH,DECIMALES_XML=DECIMALES_XML,XML_temp=XML_temp,MAX_WIDTH_ARROWS=MAX_WIDTH_ARROWS,PLOT_CHANGE=PLOT_CHANGE,DATOch=DATOch,
INCREASE_COLOR=INCREASE_COLOR,DECREASE_COLOR=DECREASE_COLOR)


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 444 to 445 in 308f210

for(ii in 1:length(LABELch)){
change(txml,LABELch[[ii]]$old,LABELch[[ii]]$new,T_ID_ARROW=T_ID_ARROW,VAL_MAX_WIDTH=VAL_MAX_WIDTH,DECIMALES_XML=DECIMALES_XML,XML_temp=XML_temp,MAX_WIDTH_ARROWS=MAX_WIDTH_ARROWS,PLOT_CHANGE=PLOT_CHANGE,DATOch=DATOch)


[air] reported by reviewdog 🐶

txml <- change(txml,"<WIDTH_MAX>",VAL_MAX_WIDTH,T_ID_ARROW=T_ID_ARROW,VAL_MAX_WIDTH=VAL_MAX_WIDTH,DECIMALES_XML=DECIMALES_XML,XML_temp=XML_temp,MAX_WIDTH_ARROWS=MAX_WIDTH_ARROWS,PLOT_CHANGE=PLOT_CHANGE,DATOch=DATOch)


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 452 to 454 in 308f210

x <- which(dact$label=="<CRPLNDTOTN>")
if(length(x)==0){
xx <- subset(dact,is.element(label,c("<PERrN>","<PERiN>","<NPErN>","<NPEiN>","<GREHN>")))


[air] reported by reviewdog 🐶

xxx <- xx %>% group_by(province,year,align,arrowColor) %>% summarise(label="<CRPLNDTOTN>",data=sum(data))


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 458 to 461 in 308f210

xxx <- xxx[,c("province","year","label","data","align","arrowColor")]
dact <- rbind(dact,xxx)
if(PLOT_CHANGE){


[air] reported by reviewdog 🐶

dactch <- rbind(dactch,xxx)


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 469 to 470 in 308f210


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 477 to 480 in 308f210

for(ii in 1:length(UNITch)){
dact$data[dact$label==UNITch[[ii]]$old] <- as.numeric(dact$data[dact$label==UNITch[[ii]]$old])/UNITch[[ii]]$div
dact$label[dact$label==UNITch[[ii]]$old] <- UNITch[[ii]]$new
dactch$label[dactch$label==UNITch[[ii]]$old] <- UNITch[[ii]]$new


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 484 to 488 in 308f210

for(lact in unique(dact$label)){
x <- dact[dact$label==lact,]
if(PLOT_CHANGE){
xch <- dactch[dactch$label==lact,]


[air] reported by reviewdog 🐶

if(length(x[,1])!=length(YEARS)){


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 494 to 495 in 308f210

if(PLOT_CHANGE){
if(length(xch[,1])!=length(YEARS_CHANGE)){


[air] reported by reviewdog 🐶

if(lact=="<PROVINCE_NAME>"){


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 510 to 516 in 308f210

}else{
if(lact=="<YEAR>"){
mean_val <- paste0("mean_",min(YEARS),"-",max(YEARS))
}else{


[air] reported by reviewdog 🐶

whep/R/GRAFS-functions.R

Lines 518 to 520 in 308f210

mean_val <- round(mean(x$data),DECIMALES_XML) #max(DECIMALES_TXT,DECIMALES_XML))
if(mean_val==0){
mean_val <- round(mean(x$data),DECIMALES_XML+1)


[air] reported by reviewdog 🐶

if(PLOT_CHANGE){


[air] reported by reviewdog 🐶

value_change <- mean(x$data)*100/mean(xch$data)-100


[air] reported by reviewdog 🐶

if(mean(xch$data)==0){ # el base era 0


[air] reported by reviewdog 🐶

txml <- change(txml,lact,mean_val,x$align[1],value_change,T_ID_ARROW=T_ID_ARROW,VAL_MAX_WIDTH=VAL_MAX_WIDTH,DECIMALES_XML=DECIMALES_XML,XML_temp=XML_temp,MAX_WIDTH_ARROWS=MAX_WIDTH_ARROWS,PLOT_CHANGE=PLOT_CHANGE,DATOch=DATOch,ARROW_COLOR=arrowcolor)


[air] reported by reviewdog 🐶


[air] reported by reviewdog 🐶

if(!PLOT_CHANGE){


[air] reported by reviewdog 🐶


[air] reported by reviewdog 🐶

crea_png(DRAW_IO_EXE,FACT_XML,str_replace_all(FACT_XML,"xml","png"))


[air] reported by reviewdog 🐶


[air] reported by reviewdog 🐶

whep/R/GRAFS_example.R

Lines 20 to 21 in 308f210

run_basic_example <- function(){


[air] reported by reviewdog 🐶

XLSX_INPUTS <- system.file("extdata", "GRAFS_spain_data.xlsx", package = "GRAFS")


[air] reported by reviewdog 🐶

TABLA_ID_XML <- system.file("extdata", "GRAFS_arrows_ids.xlsx", package = "GRAFS")


[air] reported by reviewdog 🐶

if(!file.exists(DRAW_IO_EXE)){


[air] reported by reviewdog 🐶

if(!file.exists(DRAW_IO_EXE)){


[air] reported by reviewdog 🐶

if(length(DRAW_IO_EXE)==0){


[air] reported by reviewdog 🐶

whep/R/GRAFS_example.R

Lines 46 to 52 in 308f210

create_GRAFS(XLSX_INPUTS,PATH_OUTPUTS,
XML_BASE,TABLA_ID_XML,
DRAW_IO_EXE,
REGIONS = "spain",
PERIODS = c("2011:2015-1990:1994")
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants