\title{Gets a fadn.raw.csv (csv file from DG-AGRI) and transforms it accordingly to fadn.raw.rds}
\usage{
convert.to.fadn.raw.rds(
file.path = "",
sepS = ",",
fadn.year = NA,
fadn.country = NA,
keep.csv = F,
col.id = "ID"
)
}
\arguments{
\item{file.path}{the full path of the csv file (the filename must be included)}
\item{sepS}{the separator of the csv files (by default ",")}
\item{fadn.year}{the year the csv files refers to (e.g. 2001)}
\item{fadn.country}{the three letter country code the csv files refers to (e.g. "ELL")}
\item{keep.csv}{if TRUE, copy the csv files to the CSV directory; else do not copy}
}
\value{
Saves the fadn.raw.rds file and returns TRUE if everything goes well
}
\description{
It saves two files:
- One that contain a wide format of the data, i.e. in tabular format that is identical to the csv data. This is uncompressed data.
- One that holds the same information in compressed data. It is a list that contains $data.char and $data.num data.tables in long format. 0 values are removed and only the col.id is the index on both data.tables
\title{Updates selected elements of data stored in one DT with new one given in melted format}
\usage{
update_elements.DT(data.old, data.new)
}
\arguments{
\item{data.old}{The DT to update}
\item{data.new}{The data to insert. It must have three columns: {id,variable,new value}. E.g. data.new=data.table("id"=c(810001100105),"variable"=c("AASBIO_CV"),value=c(999999))}
}
\value{
a DT with the updated values
}
\description{
The user provides the data.new: {id,variable,new value}. The function overwrites all existing id-column with the new values
\title{Merges two raw_str_map files and returns either a list or a file}
\usage{
raw_str_map.merge(
source.raw_str_map.file = NULL,
new.raw_str_map.file = NULL,
return.file = F
)
}
\arguments{
\item{source.raw_str_map.file}{the filename of the source raw_str_map. It must be relative the raw_str_maps of the current data.dir}
\item{new.raw_str_map.file}{the filename of the mask raw_str_map. It will replace any entries of the source file. It must be relative the raw_str_maps of the current data.dir}
\item{return.file}{If set to T, a temporary full file path that contains the merge is returned. Otherwise a list with the contents of the merge is returned}
}
\value{
FALSE in case of problem / if return.file=T, the temporary full path of a file that contains the merged result in json / A list with the contents of the merge if return.file=F
}
\description{
All entries in the new.raw_str_map file replace those on the source.raw_str_map file
}
\details{
Both files must be relative to the current data.dir/raw_str_maps
\title{Converts an fadn.raw.rds file to fadn.str.rds file using a raw_str_map.json file}
\usage{
convert.to.fadn.str.rds(
fadn.country = NA,
fadn.year = NA,
raw_str_map.file = NULL,
force_external_raw_str_map = FALSE,
str.name = NULL,
DEBUG = F
)
}
\arguments{
\item{fadn.country}{string with the country to extract the str data}
\item{fadn.year}{the year to extract the structured data}
\item{raw_str_map.file}{the full path to the raw_str_map file.}
\item{DEBUG}{if TRUE, prints more details on the conversion process}
\item{str.short_name}{the short name of the str data. No spaces and text up to 20 characters}
}
\value{
Saves the rds.str.fadn and returns TRUE if everything goes well
}
\description{
The raw_str_map.json specification is as follows:
}
\details{
{
"id": { "COLUMN in every list member in RDS": "COLUMN IN CSV", ....},
"info": { "COLUMN in info RDS": "COLUMN IN CSV", ....},
"livestock": {}
"crops": {
"CROP NAME 1": {"description": "description of crop name", "columns": {"VARIABLE NAME": COLUMN IN CSV", ....} },
"CROP NAME 2": {"description": "description of crop name", "columns": {"VARIABLE NAME": COLUMN IN CSV", ....} },
....
}
}
The structure of the str.dir:
- A data.dir can hold more than one extractions.
- Each extraction has a short name (20 or less characters, whitespace is not allowed)
- Each extraction is stored in the data.dir/rds/<extraction_name>
- That folder contains the following files:
+ raw_str_map.json: the raw_str_map
+ fadn.str.<4-digit YEAR>.<3-letter COUNTRY>.rds: the extracted data
Notes:
1) The computed RDS file contains a list structure with the following keys: info, costs, livestock-animals and crops
All are data.tables. For all of them, the first columns are those that are contained in the "id" object
"info" and "costs" are in table format, i.e. each farm is one row and data is on columns, as defined in the
related raw_str_map.json file.
"crops" and "livestock-animals" are in wide data format (https://tidyr.tidyverse.org/), where one farm lies accross many rows, and each
row is a farm-crop-variableName-value combination
2) In $id, $info and $costs, "COLUMN IN CSV" can have two forms
i) a single column name in the fadn.raw csv file or a combination, e.g. "K120SA+K120FC+K120FU+K120CV-K120BV"
ii) the form of an object {"source": "the column in the csv", "description": "a description of what this column is about"}
3) We attach certain attributes that are useful for identifying informations:
i) In $info and $costs, the attribute "column description" provide information of the formula and the description of each column
ii) In $crops and $livestock-animals, the attribute "$crops.descriptions" and "$livestock.descriptions", provide the description of each CROP contained there
iii) In $crops and $ the attribute "$column.formulas" provide the formulas used in order to derive the VALUE