Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
export(checksum)
export(convert_base4frac_to_dec)
export(convert_dec_to_base4frac)
export(convert_line_endings)
export(download_zenodo)
export(expand_types)
export(fileman_folders)
Expand Down
2 changes: 2 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ It reduces the `watersurfaces_types` element of the list output to unique combin
- `expand_types()` gains an argument `mark` (#198).
If set as `TRUE`, the logical columns `has_been_expanded` and `added_by_expansion` are added.
These mark rows as origin or destination of type expansion, respectively.
- New utility `convert_line_endings()` to maintain text file integrity (checksums) across platforms (#208).
See its documentation for advice on line endings management in combination with a distributed version control system like Git, or with text file generating functions such as `git2rdata::write_vc()`.

## Support for interim data source versions

Expand Down
119 changes: 119 additions & 0 deletions R/filemanagement.R
Original file line number Diff line number Diff line change
Expand Up @@ -558,3 +558,122 @@ assert_that_allfiles_exist <- function(x) {
)
)
}






#' Convert line endings of text files
#'
#' Converts line endings of text files from CRLF (`\r\n`) to LF (`\n`) or the
#' reverse.
#'
#' The conversion to LF is especially helpful to maintain file integrity across
#' platforms in combination with a distributed version control system like
#' `git`.
#'
#' To prevent unneeded rewriting, the `"to_lf"` direction is only executed in
#' Windows systems, unless `force = TRUE`.
#'
#' The function is designed so that it can take the output of
#' [git2rdata::write_vc()] as input in a pipeline; still the `root` argument may
#' need to be repeated in such case.
#'
#' @note The function borrows from `TAF::dos2unix()` and `TAF::unix2dos()`,
#' which work on single files.
#'
#' @section Git configuration to prevent auto-replacement of line endings:
#'
#' To prevent `git` from automatically replacing line endings in specific files
#' when checking out new versions of those files, set LF as the required line
#' ending of specific file types in a `.gitattributes` file in the root of the
#' git repository, and commit this file so that collaborators use the same
#' setting. An example line in `.gitattributes` looks like this (this example
#' makes line endings of yml files always use LF):
#'
#' `*.yml text eol=lf`
#'
#' Note that, with this setting, to have existing files updated by `git` they
#' first need to be removed from the working directory, after which you can do a
#' checkout again for these files.
#'
#' @param files Character vector of file paths; these may be relative to `root`.
#' @param direction Whether conversion is to LF (`"to_lf"`) or to CLRF
#' (`"to_crlf"`).
#' @param root An optional directory path; if present the `files` argument is
#' considered relative to `root`.
#' @param force Logical; `TRUE` will only have effect in non-Windows
#' systems. See Details.
#' @param silent Logical. Whether to print the return value to the console.
#'
#' @returns The `files` argument; invisibly unless `silent = FALSE`.
#'
#' @md
#'
#' @family functions regarding file management for N2KHAB projects
#'
#' @importFrom assertthat
#' assert_that
#' is.string
#' noNA
#' is.flag
#'
#' @examples
#' \dontrun{
#' files <- c(file1, file2)
#' convert_line_endings(files)
#' }
#'
#'
#' @export
convert_line_endings <- function(files,
direction = c("to_lf", "to_crlf"),
root = NULL,
force = FALSE,
silent = TRUE) {
assert_that(is.character(files))
assert_that(is.flag(force), noNA(force))
assert_that(is.flag(silent), noNA(silent))
if (!is.null(root)) {
assert_that(is.string(root))
assert_that(dir.exists(root))
files_def <- file.path(root, files)
} else {
files_def <- files
}
assert_that(all(file.exists(files_def)))
direction <- match.arg(direction)
# don't rewrite LF in non-Windows systems since it's already there, unless
# force is TRUE
if (!force && direction == "to_lf" && .Platform$OS.type != "windows") {
if (silent) {
return(invisible(files))
} else {
return(files)
}

}
# treat warnings from readLines() as errors:
owarn <- options(warn = 2)
on.exit(options(owarn))

for (i in files_def) {
txt <- try(readLines(i), silent = TRUE)
if (inherits(txt, "try-error")) {
stop(i, " is not a standard text file.")
}
con <- file(i, open = "wb")
if (direction == "to_lf") {
writeLines(txt, con, sep = "\n")
} else if (direction == "to_crlf") {
writeLines(txt, con, sep = "\r\n")
}
close(con)
}
if (silent) {
return(invisible(files))
} else {
return(files)
}
}
1 change: 1 addition & 0 deletions man/checksum.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

86 changes: 86 additions & 0 deletions man/convert_line_endings.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/download_zenodo.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/fileman_folders.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/fileman_up.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/get_zenodo_versions.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/locate_n2khab_data.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions pkgdown/_pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@ reference:
- types
- env_pressures
- namelist
- title: "Using the raw spatial data sources of habitats/RIBs"
- title: "Using the raw data sources of habitats/RIBs"
contents:
- read_favenv
- read_habitatmap
- read_habitatstreams
- read_habitatsprings
- read_habitatquarries
- title: "Using the processed spatial data sources of habitats/RIBs"
- title: "Using the processed data sources of habitats/RIBs"
contents:
- read_habitatmap_stdized
- read_habitatmap_terr
Expand All @@ -49,9 +50,11 @@ reference:
- title: "Managing files and directories"
contents:
- download_zenodo
- get_zenodo_versions
- locate_n2khab_data
- fileman_folders
- fileman_up
- convert_line_endings
- title: "Utilities and helpers"
contents:
- read_namelist
Expand Down
Loading