Let’s start by creating a simple RDO that will be a dependency for another RDO.
data_mtcars <- RDO::RDO$new(name = "data_mtcars")
data_mtcars$code <- expression({
data_mtcars <- mtcars
})Now let’s create a new RDO with R code that needs the data_mtcars object to be evaluated first.
mtcars_half_top <- RDO::RDO$new(name = "mtcars_half_top")
mtcars_half_top$code <- expression({
mtcars_half_top <- head(data_mtcars, 16)
})The code needs data_mtcars as dependency but no dependency is defined yet.
Running the RDO code will give an error as the dependency is missing.
mtcars_half_top$run()
#> Evaluating RDO: mtcars_half_top ...
#> Error while running RDO code!
#> Always check for missing dependencies...
#> Error in x[seq_len(n)]: object of type 'environment' is not subsettableWe can add add the missing dependency to the RDO.
mtcars_half_top$add_dependencies(dependencies = data_mtcars)
mtcars_half_top$run()
#> Evaluating RDO: mtcars_half_top ... has dependencies ...
#> RDO: 'data_mtcars' is NOT VALIDATED!
#> Error in .f(.x[[i]], ...): Dependency object 'data_mtcars' is not validated.Running the code still gives an error. This time the dependency is present but is not validated. Let’s run the dependency code first.
data_mtcars$run()
#> Evaluating RDO: data_mtcars ... done!
#> ...evaluation of data_mtcars completed.
mtcars_half_top$run()
#> Evaluating RDO: mtcars_half_top ... has dependencies ...
#> RDO: 'data_mtcars' is validated.
#> ...evaluation of mtcars_half_top completed.
NROW(mtcars_half_top$cache)
#> [1] 16
mtcars_half_top$code
#> <RDO code>
#> mtcars_half_top <- head(data_mtcars, 16)
#> <RDO>
#> Name: mtcars_half_top
#> Dependencies:
#> - data_mtcars
#> Status:
#> - created: 2020-03-09 13:59:52 UTC
#> - last changed: 2020-03-09 13:59:53 UTC
#> - last touched: 2020-03-09 13:59:53 UTC
#> - last validated: 2020-03-09 13:59:53 UTC
#> - last run time: 0.0180010795593262
#> - run time total: 0.0190012454986572
#> - is validated? TRUE
#> - is locked? FALSE
#> - cache size: 0.0044 Mb
#> - cached total: 0.0113 MbWe can see that our mtcars_half_top RDO has data_mtcars RDO as dependency. The object is validated, and the cache contains top 16 rows of mtcars dataset.
Let’s define more RDOs with respective R code. We can use dependencies argument of the constructor function to set dependencies in one step.
mtcars_half_bottom <-
RDO::RDO$new(name = "mtcars_half_bottom",
dependencies = list(data_mtcars))
mtcars_half_bottom$code <- expression({
mtcars_half_bottom <- tail(data_mtcars, 16)
})
mtcars_whole <-
RDO::RDO$new(name = "mtcars_whole",
dependencies = list(mtcars_half_top,
mtcars_half_bottom))
mtcars_whole$code <- expression({
mtcars_whole <- rbind(mtcars_half_top,
mtcars_half_bottom)
})
data_iris <-
RDO::RDO$new(name = "data_iris")
data_iris$code <- expression({
data_iris <- iris
})
iris_selected_rows <-
RDO::RDO$new(name = "iris_selected_rows",
dependencies = list(data_iris))
iris_selected_rows$code <- expression({
set.seed(1234)
sampled_rows <- c(rep(TRUE, 32), rep(FALSE, NROW(data_iris) - 32))
iris_selected_rows <- data_iris[sample(sampled_rows, NROW(data_iris)), ]
})
iris_selected_columns <-
RDO::RDO$new(name = "iris_selected_columns",
dependencies = list(iris_selected_rows))
iris_selected_columns$code <- expression({
iris_selected_columns <- iris_selected_rows[, c("Species", "Petal.Width")]
})
iris_mtcars <-
RDO::RDO$new(name = "iris_mtcars",
dependencies = list(mtcars_whole,
iris_selected_columns))
iris_mtcars$code <- expression({
iris_mtcars <- cbind(iris_selected_columns, mtcars_whole)
})
iris_mtcars_test <-
RDO::RDO$new(name = "iris_mtcars_test",
dependencies = list(iris_mtcars))
iris_mtcars_test$code <- expression({
stopifnot("mpg" %in% names(iris_mtcars))
iris_mtcars <- iris_mtcars
})Most RDO methods has deep parameter. W hen set to FALSE (default) you won’t be interacting with RDO dependencies. If set to TRUE the method will touch all deep dependencies inside your RDO. By deep dependencies I mean all dependencies of the RDO, their dependencies, and dependencies of their dependencies, etc.
Let’s take the iris_mtcars_test RDO that is at the top of our RDO tree, and see the code needed to reproduce the whole tree.
iris_mtcars_test$print_code(deep = TRUE)
#> data_mtcars <- mtcars
#> mtcars_half_top <- head(data_mtcars, 16)
#> mtcars_half_bottom <- tail(data_mtcars, 16)
#> data_iris <- iris
#> set.seed(1234)
#> sampled_rows <- c(rep(TRUE, 32), rep(FALSE, NROW(data_iris) - 32))
#> iris_selected_rows <- data_iris[sample(sampled_rows, NROW(data_iris)), ]
#> mtcars_whole <- rbind(mtcars_half_top, mtcars_half_bottom)
#> iris_selected_columns <- iris_selected_rows[, c("Species", "Petal.Width")]
#> iris_mtcars <- cbind(iris_selected_columns, mtcars_whole)
#> stopifnot("mpg" %in% names(iris_mtcars))
#> iris_mtcars <- iris_mtcarsWe can check what dependencies are validated.
iris_mtcars_test$is_validated(deep = TRUE)
#> RDO: 'data_mtcars' is validated.
#> RDO: 'mtcars_half_top' is validated.
#> RDO: 'mtcars_half_bottom' is NOT VALIDATED!
#> RDO: 'data_iris' is NOT VALIDATED!
#> RDO: 'iris_selected_rows' is NOT VALIDATED!
#> RDO: 'mtcars_whole' is NOT VALIDATED!
#> RDO: 'iris_selected_columns' is NOT VALIDATED!
#> RDO: 'iris_mtcars' is NOT VALIDATED!
#> RDO: 'iris_mtcars_test' is NOT VALIDATED!
#> [1] FALSELet’s run the code with deep = TRUE. It will run the code of all not validated RDOs.
iris_mtcars_test$run(deep = TRUE)
#> Evaluating RDO: iris_mtcars_test ... has dependencies ...
#> RDO: 'data_mtcars' is validated.
#> RDO: 'mtcars_half_top' is validated.
#> RDO: 'mtcars_half_bottom' is NOT VALIDATED!
#> Evaluating RDO: mtcars_half_bottom ... has dependencies ...
#> RDO: 'data_mtcars' is validated.
#> ...evaluation of mtcars_half_bottom completed.
#> RDO: 'data_iris' is NOT VALIDATED!
#> Evaluating RDO: data_iris ... done!
#> ...evaluation of data_iris completed.
#> RDO: 'iris_selected_rows' is NOT VALIDATED!
#> Evaluating RDO: iris_selected_rows ... has dependencies ...
#> RDO: 'data_iris' is validated.
#> ...evaluation of iris_selected_rows completed.
#> RDO: 'mtcars_whole' is NOT VALIDATED!
#> Evaluating RDO: mtcars_whole ... has dependencies ...
#> RDO: 'mtcars_half_top' is validated.
#> RDO: 'mtcars_half_bottom' is validated.
#> ...evaluation of mtcars_whole completed.
#> RDO: 'iris_selected_columns' is NOT VALIDATED!
#> Evaluating RDO: iris_selected_columns ... has dependencies ...
#> RDO: 'iris_selected_rows' is validated.
#> ...evaluation of iris_selected_columns completed.
#> RDO: 'iris_mtcars' is NOT VALIDATED!
#> Evaluating RDO: iris_mtcars ... has dependencies ...
#> RDO: 'mtcars_whole' is validated.
#> RDO: 'iris_selected_columns' is validated.
#> ...evaluation of iris_mtcars completed.
#> RDO: 'iris_mtcars' is validated.
#> ...evaluation of iris_mtcars_test completed.
iris_mtcars_test$is_validated(deep = TRUE)
#> RDO: 'data_mtcars' is validated.
#> RDO: 'mtcars_half_top' is validated.
#> RDO: 'mtcars_half_bottom' is validated.
#> RDO: 'data_iris' is validated.
#> RDO: 'iris_selected_rows' is validated.
#> RDO: 'mtcars_whole' is validated.
#> RDO: 'iris_selected_columns' is validated.
#> RDO: 'iris_mtcars' is validated.
#> RDO: 'iris_mtcars_test' is validated.
#> [1] TRUENow the iris_mtcars_test RDO is deeply validated as all its dependencies are validated as well.
We can access any RDO dependency with get_dependencies() method. With deep = TRUE the method returns a list of all deep unique dependencies. Each RDO on this dependency list can be accessed directly.
names(iris_mtcars_test$get_dependencies(deep = TRUE))
#> [1] "data_mtcars" "mtcars_half_top" "mtcars_half_bottom"
#> [4] "data_iris" "iris_selected_rows" "mtcars_whole"
#> [7] "iris_selected_columns" "iris_mtcars"
iris_mtcars_test$get_dependencies(deep = TRUE)$data_mtcars$code
#> <RDO code>
#> data_mtcars <- mtcars
#> <RDO>
#> Name: data_mtcars
#> Dependencies: none.
#> Status:
#> - created: 2020-03-09 13:59:52 UTC
#> - last changed: 2020-03-09 13:59:52 UTC
#> - last touched: 2020-03-09 13:59:53 UTC
#> - last validated: 2020-03-09 13:59:52 UTC
#> - last run time: 0.00100016593933105
#> - run time total: 0.00100016593933105
#> - is validated? TRUE
#> - is locked? FALSE
#> - cache size: 0.0069 Mb
#> - cached total: 0.0069 MbIf we don’t need to keep the data cache in all dependencies we can remove it using prune_cache() method. The cache won’t be removed from locked RDOs.
Let’s lock some RDOs and clear cache in the rest.
as.numeric(iris_mtcars_test$get_status()$cache_size_total)
#> [1] 48256
data_mtcars$lock()$is_locked()
#> Locking RDO: 'data_mtcars' ... done.
#> RDO: 'data_mtcars' is LOCKED!
#> [1] TRUE
data_iris$lock()$is_locked()
#> Locking RDO: 'data_iris' ... done.
#> RDO: 'data_iris' is LOCKED!
#> [1] TRUE
iris_mtcars_test$prune_cache(deep = TRUE)
#> RDO: 'data_mtcars' is locked!
#> Cache in RDO: 'mtcars_half_top' was cleared.
#> Cache in RDO: 'mtcars_half_bottom' was cleared.
#> RDO: 'data_iris' is locked!
#> Cache in RDO: 'iris_selected_rows' was cleared.
#> Cache in RDO: 'mtcars_whole' was cleared.
#> Cache in RDO: 'iris_selected_columns' was cleared.
#> Cache in RDO: 'iris_mtcars' was cleared.
as.numeric(iris_mtcars_test$get_status()$cache_size_total)
#> [1] 20608The cache from unlocked RDOs was removed and we reduced the total cache size.