Running a targets pipeline on RSConnect

[dewoller]

We run scheduled targets pipelines on RSConect. First, the pipeline is tested locally, and once it is ready, it is published to RSConnect, using a the following shim make_target.Rmd to run it.

---
title:           "Generate Target"
output:          html_document
---

<FILLER> ```{r setup, include = FALSE}
library(targets)
<FILLER> ```

 # Pipeline

<FILLER> ```{r make_all_the_targets}
targets::tar_make()
<FILLER> ```

When the pipeline runs successfully on both the local and RSConnect environments, this is good.

When there is an error that only occurs in the RSConnect environment, the error message does not specify which target failed. Specifically, the relevant error message is:

[Connect] 2022/08/09 01:50:24.990906807 label: make_all_the_targets
[Connect] 2022/08/09 02:32:40.687213351 Quitting from lines 16-17 (make_target.Rmd)
[Connect] 2022/08/09 02:32:40.809619106 Quitting from lines 16-17 (make_target.Rmd)
[Connect] 2022/08/09 02:32:40.809652906 Error in `$<-.data.frame`(`*tmp*`, "call_text", value = c("targets::tar_make()",  :
[Connect] 2022/08/09 02:32:40.809753006   replacement has 9 rows, data has 7
[Connect] 2022/08/09 02:32:40.809757606 Calls: local ... trace_format_branch -> trace_as_tree -> $<- -> $<-.data.frame
[Connect] 2022/08/09 02:32:40.810509306
[Connect] 2022/08/09 02:32:40.811230106 Execution halted
[Connect] 2022/08/09 02:32:40.809912806 Error code: render-r-code-error

This comes, for example, when an environment variable is defined incorrectly on the RSConnect server.

Is there a way to call the tar_make that will improve the error message? Or to at least find out which target is failing?

[wlandau]

Maybe you can point the data store to somewhere that you can download after an attempted run? From https://solutions.rstudio.com/r/workflows/, it looks like you can send it to something like /mnt/shared-data/targets-test/.

[wlandau]

Downloading the data store allows you, for example, to get information from the data store with tar_meta(fields = error, complete_only = TRUE).

[dewoller]

That is just great, like two of my best friends just met and liked each other. It is good that there is a way that RSConnect and targets can get along better, my way was a bit brute force. That should help a lot

BTW, a minimal way to cause that uninformative error message is by publishing the following erroneous _targets.R to Rsconnect, using for example the Rmd wrapper shim above.

library(targets)
list(
         tar_target( a,b)
)

I'm guessing it has something to do with a limitation of Rmarkdown, as transmuted by RSConnect, so not an easy problem to fix.

[al-obrien]

Just an FYI - unfortunately it appears the aforementioned solutions link has been deprecated ... not sure if a replacement is forthcoming but it was a very helpful reference. I have not found an equivalent replacement in the Posit manuals.