Background
replace resources truncate data and their state each time pipeline runs. historically they were also used to perform full refresh by overwriting the write disposition on pipeline level and then going back to standard resource definition.
The side effect of this method is that participating tables are really switched to replace write disposition and such schema is propagated and stored. Until data is loaded again the pipeline state is in "replace" mode.
There's a way better and cleaner method that does not touch resource definition: using refresh argument. This PR will remove all mentions of the old methods from docs, add additional tests and also restore correct truncation behavior for replace write disposition that were reverted by #3998
Background
replaceresources truncate data and their state each time pipeline runs. historically they were also used to perform full refresh by overwriting the write disposition on pipeline level and then going back to standard resource definition.The side effect of this method is that participating tables are really switched to
replacewrite disposition and such schema is propagated and stored. Until data is loaded again the pipeline state is in "replace" mode.There's a way better and cleaner method that does not touch resource definition: using
refreshargument. This PR will remove all mentions of the old methods from docs, add additional tests and also restore correct truncation behavior forreplacewrite disposition that were reverted by #3998