upstream merge template repository

.env

@@ -1,7 +1,7 @@
 # development settings
 CONFIGURATION_FILE=./app-configuration.json
 PRINT_CONFIGURATION=yes
-SOURCE_FILE=./data/raw/yahatinda1.rds
+SOURCE_FILE=./data/raw/input4_move2loc_LatLon.rds
 OUTPUT_FILE=./data/output/output.rds
 ERROR_FILE=./data/output/error.log
 APP_ARTIFACTS_DIR=./data/output/

.gitignore

@@ -2,4 +2,7 @@
 .Rhistory
 .RData
 .Ruserdata
-data/output
+.DS_Store
+data/output/**
+!data/output/.keep
+tests/testthat/plot.png

CHANGELOG_SDK.md

@@ -0,0 +1,55 @@
+# Changelog SDK
+## XXX XXX
+- update of `README.md` including new sections
+- update of `developer_README.md`
+
+## 2024-03 `v3.2.0`
+
+- introduce app-setting-type `USER_FILE`
+- deprecate app-setting-type `LOCAL_FILE`
+
+## 2024-02 `v3.1.0`
+
+- introduce `appspec.json` version `1.2`
+    - removed `createsArtifacts`. It is safe to remove it completely from your `appspec.json` - MoveApps tries now to fetch artifacts for every running App.
+    - verify to include the `null` option for setting types `DROPDOWN` and `RADIOBUTTONS` if `defaultValue` is set to `null`.
+- drop `move1` dependency
+
+## 2023-12 `v3.0.5`
+
+- update of input files for testing apps. Files now include `move2_loc` and `telemetry.list` I/O types, and projected and not projected data. All details included in the README.txt in the data/raw folder.
+
+## 2023-11 `v3.0.4`
+
+- Upgrade `R` framework to `4.3.2`
+
+## 2023-09 `v3.0.3`
+
+- introduce `appspec.json` version `1.1`
+    - documentation link is not required any more
+
+## 2023-08 `v3.0.2`
+
+- removed MoveApps IO-Type dependency
+
+## 2023-07 `v3.0.1`
+
+- provide new `move2` input files
+- minor fixes in src/io files
+
+## 2023-06 `v3.0.0`
+
+- introduces template versioning (starting w/ `v3.0.0` as this is the third major iteration)
+- introduces `dotenv` to control/adjust local app-development
+- updates developer readme
+- introduces a _Template Synchronization_ GH action. Use it to synchronize your forked app with template updates. If you already forked from the template _before_ SDK `v3.0.0` you can (*only use this option if your app is written for input data of class `move2`*): 
+    1. manually add the files `.github/workflows/template-sync.yml` and `.github/.templatesyncignore` to your fork. 
+    1. with these files you can manually execute the GH action named _.github/workflows/template-sync.yml_. 
+    1. after merging the generated pull request add the file `app-configuration.json` manually to the root directory of your App.
+
+- fix app-configuration for execution on moveapps.org
+- clear app output of previously app run at each start of the SDK
+
+- SDK supports [`move2`](https://gitlab.com/bartk/move2/), if input is of class [`move`](https://gitlab.com/bartk/move/) it will be converted into class `move2`.
+- output is always move2
+- Upgrade `R` framework to `4.3.1`

Dockerfile

@@ -1,8 +1,8 @@
 ########################################################################################################################
-# MoveApps R SDK aka co-pilot-r
+# MoveApps R SDK
 ########################################################################################################################
 
-FROM rocker/geospatial:4.2.1
+FROM rocker/geospatial:4.4.2
 
 LABEL maintainer = "couchbits GmbH <us@couchbits.com>"
 
@@ -16,19 +16,14 @@ USER moveapps:staff
 
 WORKDIR /home/moveapps/co-pilot-r
 
-# renv
-ENV RENV_VERSION 0.15.5
-RUN R -e "install.packages('remotes', repos = c(CRAN = 'https://cloud.r-project.org'))"
-RUN R -e "remotes::install_github('rstudio/renv@${RENV_VERSION}')"
-COPY --chown=moveapps:staff renv.lock .Rprofile ./
-COPY --chown=moveapps:staff renv/activate.R renv/settings.dcf ./renv/
-
 # copy the SDK
 COPY --chown=moveapps:staff src/ ./src/
 COPY --chown=moveapps:staff data/ ./data/
-COPY --chown=moveapps:staff co-pilot-sdk.R RFunction.R start-process.sh ./
-RUN mkdir ./data/output
-# and restore the R libraries
+COPY --chown=moveapps:staff sdk.R RFunction.R .env app-configuration.json start-process.sh ./
+
+# restore the current snapshot via renv
+COPY --chown=moveapps:staff renv.lock .Rprofile ./
+COPY --chown=moveapps:staff renv/activate.R renv/settings.dcf ./renv/
 RUN R -e 'renv::restore()'
 
 ENTRYPOINT ["/bin/bash"]

data/raw/README.txt

@@ -0,0 +1,37 @@
+Set of input data to test apps.
+
+*Content*
+- input1: 1 goat, median fix rate = 30mins, tracking duration 7.5 month, gps, local movement
+- input2: 3 storks, median fix rate = 1sec, tracking duration 2 weeks, gps, local movement
+- input3: 1 stork, median fix rate = 1h | 1day | 1 week, tracking duration 11.5 years, argos, includes migration
+- input4: 3 geese, median fix rate = 1h | 4h, tracking duration 1.5 years, gps, includes migration
+
+*I/O types*
+- all data sets are provided as 'move2_loc'
+- input1 & input2 are also provided as 'telemetry.list'
+
+*Projection*
+- for 'move2_loc', data are provided in "lat/long" (EPSG:4326) and projected to "Mollweide" (ESRI:54009) in order to test your app accordingly for not projected and projected data. If your app does not allow projected data or only can deal with projected data, document and either build a automatic transformation in the app or make it fail with an informative error message. The app "Change projection" can be refered to for the user to change the projection of the data acordingly previous to your app.
+
+- the 'telemetry.list' examples are in a "aeqd" projection with 0,0 in the center of the track, as this is a common projection used within the ctmm library
+
+
+*File names*
+input1_move2loc_LatLon.rds
+input1_move2loc_Mollweide.rds
+
+input2_move2loc_LatLon.rds
+input2_move2loc_Mollweide.rds
+
+input3_move2loc_LatLon.rds
+input3_move2loc_Mollweide.rds
+
+input4_move2loc_LatLon.rds
+input4_move2loc_Mollweide.rds
+
+
+input1_telemetrylist_aeqd.rds
+input2_telemetrylist_aeqd.rds
+
+
+

developer_README.md

@@ -1,29 +1,86 @@
-# Developing with the MoveApps SDK
+# MoveApps R Software Development Kit (SDK)
 
-### General notes
+***NOTE*: this SDK supports code written for input data of class `move2` and not `moveStack`, as all input data of class `moveStack` will be converted to class `move2`. For all other input/output types, this SDK works as usual. Please contact us under support@moveapps.org if you have any questions.**
 
-- get an overview with the help of the [user manual](https://docs.moveapps.org/#/create_app)
-- files needed for your app:
-  - your app codes goes to `./RFunction.R`
-  - setup your app arguments and your environment by adjusting `./appspec.json`
-  - the documentation of your app goes to `./README.md`
-- to run and test your app code locally in a simulated MoveApps environment adjust and execute the file `./co-pilot-sdk.R`
-  - adjust the `inputFileName`
-  - state the arguments of your function if present
+This documentation provides a short introduction to the [MoveApps](https://www.moveapps.org) **R SDK**.
 
-### R packages management (optional)
+As a first step, and before your read this, you should have used this GitHub template to create a copy of it in your personal space and named the repository as your App will be named in MoveApps.
 
-The template is prepared to use [`renv` as a dependency manager](https://rstudio.github.io/renv/articles/renv.html) - but is disabled ("opt-in") by default.    
+**The [MoveApps User Manual](https://docs.moveapps.org/#/create_app) provides a step-by-step explanation of how to create an App.** Please carefully follow these steps when creating a MoveApps App.
+
+
+## Files in the SDK/template
+
+This template is designed according to a file structure that is necessary for your App to run in your local development environment similar to the way it will run in the MoveApps environment later. Please contain the structure and only change/add files as necessary for your App's functionality. Take a look at the [overview in the User Manual](https://docs.moveapps.org/#/create_app) to see which files can be changed and which should remain as is for simulation of the behaviour on MoveApps on your local system.
+
+Here you find an overview of the files and their function in the SDK:
+
+1. `./RFunction.R`: must be modified by the developer. This is the entrypoint for your App logic. MoveApps will call this function during a Workflow execution which includes your App. The file must be named `RFunction.R`, do not alter it. See [Step 3](https://docs.moveapps.org/#/create_app#step-3-develop-the-app-code-locally-within-the-template) in the User Manual.
+1. `./appspec.json`: must be modified by the developer. This file defines the settings and metadata of your App. See [Step 5](https://docs.moveapps.org/#/create_app#step-5-write-app-specifications) in the User Manual.
+1. `./README.md`: must be modified by the developer. Provided template for the documentation of the App (see [Step 6](https://docs.moveapps.org/#/create_app#step-6-write-a-documentation-file) in the User Manual).
+1. `./tests/**`: must be modified by the developer. Location for *Unit Tests*. See [Step 4](https://docs.moveapps.org/#/create_app#=step-4-test-your-app-locally) of the User Manual.
+1. `./renv.lock`: Definition of the dependencies of your App. We use `renv` as library manager. Optional, see below.
+1. `./sdk.R`: use for App testing. The main entrypoint of the SDK. Use it to execute your App in your compiler (e.g. RStudio).
+1. `./app-configuration.json`: adjust for App testing. The configuration/settings file of your App (in [JSON](https://www.w3schools.com/js/js_json_intro.asp) format - must correspondent with the `settings` of your `appspec.json`, see [MoveApps parameters](https://docs.moveapps.org/#/copilot-r-sdk.md#moveapps-parameters) for an example).
+1. `/.env`: adjust for App testing. Defining the SDK Runtime environment, see below. Make sure to check _Show Hidden Files_ in the settings menu of the _Files_ tab in RStudio.
+1. `./data/**`: use for App testing. Resources of the SDK
+   1. `auxiliary/**`: Simulates the usage of [*auxiliary files*](https://docs.moveapps.org/#/auxiliary). You can put files into this folder to simulate an App run with provided/user-uploaded files. 
+   1. `output/**`: The output data (`output.rds`) that will be passed on to the next App in a Workflow and other output files (artifacts) that your App may produce will be stored here. See [*producing artifacts*](https://docs.moveapps.org/#/copilot-r-sdk?id=producing-artefacts) for more information.
+   1. `raw/**`: Collection of sample App input data. You can use these data to test an App run with real input.
+1. all remaining files are used to emulate MoveApps when testing the App locally, to setup the automatic sync of your repository with the template, or to provide information related to the template. **These files are not to be modified**. Modifying them will prevent you from testing your App appropriately. 
+
+## SDK runtime environment
+
+Critical parts of the SDK can be adjusted by `environment variables`. Keep in mind that these variables are only changeable during App development and not during an App run on MoveApps. They are predefined with sensible defaults - they should work for you as they are. While testing your App you will want to modify the `SOURCE_FILE` variable to either call the different example data sets provided in the template or other data sets that you want to use to test your App.
+
+- `CONFIGURATION_FILE`: path to the configuration/settings file of your App (in [JSON](https://www.w3schools.com/js/js_json_intro.asp) format - must correspondent with the `settings` of your `appspec.json`, see [MoveApps parameters](https://docs.moveapps.org/#/copilot-r-sdk.md#moveapps-parameters) for an example of the `app-configuration.json` file).
+- `PRINT_CONFIGURATION`: prints the configuration your App receives (`yes|no`)
+- `SOURCE_FILE`: path to an input file for your App during development
+- `OUTPUT_FILE`: path to the output file of your App
+- `ERROR_FILE`: path to a file collecting error messages
+- `APP_ARTIFACTS_DIR`: base directory for writing App artifacts
+- `USER_APP_FILE_HOME_DIR`: home aka base directory of your local user/auxiliary App files
+- `CLEAR_OUTPUT`: clears all output of the previously app run at each start of the SDK aka the next app start
+
+You can adjust these environment variables by adjusting the file `./.env`.
+
+The file `./.env` is **hidden** by default in `RStudio`! You can show it by
+
+1. Tab `Files` in `RStudio`
+1. `More` Button in the Files-Toolbar
+1. Activate _Show Hidden Files_
+
+## MoveApps App Bundle
+
+Which files will be bundled into the final App running on MoveApps?
+
+- the file `./RFunction.R
+- all directories defined in your `appspec.json` at `providedAppFiles`
+
+- the file `./appspec.json` will be used to build and create the metadata of your App
+- the file `./README.md` will be reference to for the documentation of your App
+
+Nothing else.
+
+
+## R packages management / renv (optional)
+
+The template is prepared to use [`renv` as a dependency manager](https://rstudio.github.io/renv/articles/renv.html) - but is disabled by default (_opt-in_).
 You can [activate `renv` with `renv::activate()`](https://rstudio.github.io/renv/articles/renv.html#uninstalling-renv) and then use it in the [usual `renv` workflow](https://rstudio.github.io/renv/articles/renv.html#workflow).
 
 
 ### Docker support (optional)
 
-- at the end your app will be executed on MoveApps in a Docker container.
-- if you like you can test your app in the almost final environment by running your app locally in a docker container:
+Your app will be executed on MoveApps in a Docker container. Specially for debugging errors that are not straight forward, it can be very useful to execute your App in a docker container. These more complex errors are often due to system libraries that need to be installed in MoveApps to run the App. The easiest way to find out which ones they are is to run the App locally in a docker container:
+
+
+1. enable `renv` (see above)
+1. set a working title for your app by `export MY_MOVEAPPS_APP=hello-world` (in your terminal)
+1. build the Docker image locally by `docker build --platform=linux/amd64 -t $MY_MOVEAPPS_APP .` (in your terminal)
+1. execute the image with `docker run --platform=linux/amd64 --rm --name $MY_MOVEAPPS_APP -it $MY_MOVEAPPS_APP` (in your terminal)
+1. you will get a `bash` terminal of the running container. There you can get a R console by `R` or simply start your app by invoking `/home/moveapps/co-pilot-r/start-process.sh` (in the `bash` of the running container)
+
+
+## Synchronisation of your App repository with this template
 
-1. add each R library you added to your app via `renv` to the docker image by adding eg. `RUN R -e 'remotes::install_version("foreach")'` to the `./Dockerfile` before `RUN R -e 'renv::restore()'`
-1. set a working title for your app by `export MY_MOVEAPPS_APP=hello-world`
-1. build the Docker image locally by `docker build -t $MY_MOVEAPPS_APP .`
-1. execute the image with `docker run --rm --name $MY_MOVEAPPS_APP -it $MY_MOVEAPPS_APP`
-1. you will get a `bash` terminal of the running container. There you can get a R console by `R` or simply start your app by invoking `/home/moveapps/co-pilot-r/start-process.sh` inside the running container.
+This template includes a _GitHub action_ to keep your copy synchronized with the original template. Take a look at the [documentation](https://docs.moveapps.org/#/manage_Rapp_github?id=keep-your-repositories-up-to-date-sync-with-templates) and make sure to keep your repository up-to-date.