targets 1.8.0

• Wrap tar_watch() UI module in bslib::page() (#1302, @kwbyron-lilly).
• Remove callr_function in tar_make_as_job() argument list.
• Ensure storage = "worker" is respected when the process of storing an object generates an error (#1304, @multimeric).
• Default to the _targets.R pattern in tar_branches() (#1306, @multimeric, @mattwarkentin).
• Remove superfluous functions and globals from metadata with tar_prune() (#1312, @benzipperer).
• Change the default workspace_on_error option to TRUE (#1310, @hadley).
• Enhance and organize the error = "stop" error message.
• Avoid saving a file in _targets/objects for error = "null". Instead, switch to a special "null" storage format class if error is "null" the target throws an error. This should allow users to more freely create new formats with tar_format() without worrying about how to handle NULL objects created by error = "null".
• Implement format = "auto" (#1311, @hadley).
• Replace pingr dependency with base::socketConnection() for local URL utilities (#1317, #1318, @Adafede).
• Implement tar_repository_cas(), tar_repository_cas_local(), and tar_repository_cas_local_gc() for content-addressable storage (#1232, #1314, @noamross).
• Add tar_format_get() to make implementing CAS systems easier.
• Implement error = "trim" in tar_target() and tar_option_set() (#1310, #1311, @hadley).
• Use the file system type to decide whether to trust time stamps (#1315, @hadley, @gaborcsardi).
• Deprecate format = "file_fast" in favor of the above (#1315).
• Deprecate trust_object_timestamps in favor of the more unified trust_timestamps in tar_option_set() (#1315).
• Print storage size of each target in verbose reporters (#1337, @psychelzh).
• Combine help files of tar_target() and tar_target_raw(). Same with tar_load() and tar_load_raw().
• Add a substitute argument to tar_format() to make it easier to write custom storage formats without metaprogramming.

targets 1.7.1

• Use bslib in tar_watch().
• Speed up target_upstream_edges() and pipeline_upstream_edges() by avoiding data frames until the last minute (17% speedup for certain kinds of large pipelines).
• Automatically set as_job to FALSE in tar_make() if rstudioapi and/or RStudio is not available.

targets 1.7.0

Invalidating changes

• Use secretbase::siphash13() instead of digest(algo = "xxhash64", serializationVersion = 3) so hashes of in-memory objects no longer depend on serialization version 3 headers (#1244, @shikokuchuo). Unfortunately, pipelines built with earlier versions of targets will need to rerun.

Other improvements

• Ensure patterns marshal properly (#1266, #1264, ropensci/geotargets#52, @Aariq, @njtierney).
• Inform and prompt the user when the pipeline was built with an old version of targets and changes to the package will cause the current work to rerun (#1244). For the tar_make*() functions, utils::menu() prompts the user to give people a chance to downgrade if necessary.
• For type safety in the internal database class, read all columns as character vectors in data.table::fread(), then convert them to the correct types afterwards.
• Add a new tar_resources_custom_format() function which can pass environment variables to customize the behavior of custom tar_format() storage formats (#1263, #1232, @Aariq, @noamross).
• Only marshal dependencies if actually sending the target to a parallel worker.

targets 1.6.0

• Modernize extras in tar_renv().
• tar_target() gains a description argument for free-form text describing what the target is about (#1230, #1235, #1236, @tjmahr).
• tar_visnetwork(), tar_glimpse(), tar_network(), tar_mermaid(), and tar_manifest() now optionally show target descriptions (#1230, #1235, #1236, @tjmahr).
• tar_described_as() is a new wrapper around tidyselect::any_of() to select specific subsets of targets based on the description rather than the name (#1136, #1196, @noamross, @mattmoo).
• Fix the documentation of the names argument (nudge users toward tidyselect expressions).
• Make assertions on the pipeline process more robust (to check if two processes are trying to access the same data store).

targets 1.5.1

• Avoid arrow-related CRAN check NOTE.
• use_targets() only writes the _targets.R script. The run.sh and run.R scripts are superseded by the as_job argument of tar_make(). Users not using the RStudio IDE can call tar_make() with callr_function = callr::r_bg to run the pipeline as a background process. tar_make_clustermq() and tar_make_future() are superseded in favor tar_make(use_crwe = TRUE), so template files are no longer written for the former automatically.

targets 1.5.0

Invalidating changes

Because of the changes below, upgrading to this version of targets will unavoidably invalidate previously built targets in existing pipelines. Your pipeline code should still work, but any targets you ran before will most likely need to rerun after the upgrade.

• In tar_seed_create(), use secretbase::sha3(x = TARGET_NAME, bits = 32L, convert = NA) to generate target seeds that are more resistant to overlapping RNG streams (#1139, @shikokuchuo). The previous approach used a less rigorous combination of digest::digest(algo = "sha512") and digets::digest2int().

Other improvements

• Update the documentation of the deployment argument of tar_target() to reflect the advent of crew (#1208, @psychelzh).
• Unset cli.num_colors on exit in tar_error() and tar_warning() (#1210, @dipterix).
• Do not try to access seconds_timeout if the crew controller is actually a controller group (#1207, wlandau/crew.cluster#35, @stemangiola, @drejom).
• tar_make() gains an as_job argument to optionally run a targets pipeline as an RStudio job.
• Bump required igraph version to 2.0.0 because igraph::get.edgelist() was deprecated in favor of igraph::as_edgelist().
• Do not dispatch targets to backlogged crew controllers (or controller groups) (#1220). Use the new push_backlog() and pop_backlog() crew methods to make this smooth.
• Make the debugger message more generic (#1223, @eliocamp).
• Throw an early and informative error from tar_make() if there is already a targets pipeline running on a local process on the same local data store. The local process is detected using the process ID and time stamp from tar_process() (with a 1.01-second tolerance for the time stamp).
• Remove pkgload::load_all() warning (#1218). Tried using .__DEVTOOLS__ but it interferes with reverse dependencies.
• Add documentation and an assertion in tar_target_raw() to let users know that iteration = "group" is invalid for dynamic targets (ones with pattern = map(...) etc.; #1226, @bmfazio).

targets 1.4.1

• Print "errored pipeline" when at least one target errors.
• Bump minimum clustermq version to 0.9.2.
• Repair the tar_debug_instructions() tips for when commands are long.
• Do not look for dependencies of primitive functions (#1200, @smwindecker, @joelnitta).

targets 1.4.0

Invalidating changes

Because of the changes below, upgrading to this version of targets will unavoidably invalidate previously built targets in existing pipelines. Your pipeline code should still work, but any targets you ran before will most likely need to rerun after the upgrade.

• Use SHA512 during the creation of target-specific pseudo-random number generator seeds (#1139). This change decreases the risk of overlapping/correlated random number generator streams. See the "RNG overlap" section of the tar_seed_create() help file for details and justification. Unfortunately, this change will invalidate all currently built targets because the seeds will be different. To avoid rerunning your whole pipeline, set cue = tar_cue(seed = FALSE) in tar_target().
• For cloud storage: instead of the hash of the local file, use the ETag for AWS S3 targets and the MD5 hash for GCP GCS targets (#1172). Sanitize with targets:::digest_chr64() in both cases before storing the result in the metadata.
• For a cloud target to be truly up to date, the hash in the metadata now needs to match the current object in the bucket, not the version recorded in the metadata (#1172). In other words, targets now tries to ensure that the up-to-date data objects in the cloud are in their newest versions. So if you roll back the metadata to an older version, you will still be able to access historical data versions with e.g. tar_read(), but the pipeline will no longer be up to date.

Other changes to seeds

• Add a new exported function tar_seed_create() which creates target-specific pseudo-random number generator seeds.
• Add an "RNG overlap" section in the tar_seed_create() help file to justify and defend how targets and tarchetypes approach pseudo-random numbers.
• Add function tar_seed_set() which sets a seed and sets all the RNG algorithms to their defaults in the R installation of the user. Each target now uses tar_seed_set() function to set its seed before running its R command (#1139).
• Deprecate tar_seed() in favor of the new tar_seed_get() function.

Other cloud storage improvements

• For all cloud targets, check hashes in batched LIST requests instead of individual HEAD requests (#1172). Dramatically speeds up the process of checking if cloud targets are up to date.
• For AWS S3 targets, tar_delete(), tar_destroy(), and tar_prune() now use efficient batched calls to delete_objects() instead of costly individual calls to delete_object() (#1171).
• Add a new verbose argument to tar_delete(), tar_destroy(), and tar_prune().
• Add a new batch_size argument to tar_delete(), tar_destroy(), and tar_prune().
• Add new arguments page_size and verbose to tar_resources_aws() (#1172).
• Add a new tar_unversion() function to remove version IDs from the metadata of cloud targets. This makes it easier to interact with just the current version of each target, as opposed to the version ID recorded in the local metadata.

Other improvements

• Migrate to the changes in clustermq 0.9.0 (@mschubert).
• In progress statuses, change "started" to "dispatched" and change "built" to "completed" (#1192).
• Deprecate tar_started() in favor of tar_dispatched() (#1192).
• Deprecate tar_built() in favor of tar_completed() (#1192).
• Console messages from reporters say "dispatched" and "completed" instead of "started" and "built" (#1192).
• The crew scheduling algorithm no longer waits on saturated controllers, and targets that are ready are greedily dispatched to crew even if all workers are busy (#1182, #1192). To appropriately set expectations for users, reporters print "dispatched (pending)" instead of "dispatched" if the task load is backlogged at the moment.
• In the crew scheduling algorithm, waiting for tasks is now a truly event-driven process and consumes 5-10x less CPU resources (#1183). Only the auto-scaling of workers uses polling (with an inexpensive default polling interval of 0.5 seconds, configurable through seconds_interval in the controller).
• Simplify stored target tracebacks.
• Print the traceback on error.

targets 1.3.2

• Try to fix function help files for CRAN.

targets 1.3.1

• Add tar_config_projects() and tar_config_yaml() (#1153, @psychelzh).
• Apply error modes to builder_wait_correct_hash() in target_conclude.tar_builder() (#1154, @gadenbuie).
• Remove duplicated error message from builder_error_null().
• Allow tar_meta_upload() and tar_meta_download() to avoid errors if one or more metadata files do not exist. Add a new argument strict to control error behavior.
• Add new arguments meta, progress, process, and crew to control individual metadata files in tar_meta_upload(), tar_meta_download(), tar_meta_sync(), and tar_meta_delete().
• Avoid newly deprecated arguments and functions in crew 0.5.0.9003 (https://github.com/wlnadau/crew/issues/131).
• Allow tar_read() etc. inside a pipeline whenever it uses a different data store (#1158, @MilesMcBain).
• Set seed = FALSE in future::future() (#1166, @svraka).
• Add a new physics argument to tar_visnetwork() and tar_glimpse() (#925, @Bdblodgett-usgs).