Datasets schedules are data load jobs that run frequently to import data from the source to your dataset, ensuring your data is always up to date. You can setup or edit a dataset schedule in two different pages
1- The data load schedule page, on the lat step of adding a dataset through a relational database connection:
2- You can also edit existing data load schedule from the dataset edit page: My datasets>edit dataset>Load Schedule
Below are the settings you can change for a data load schedule:
| Setting | Explanation |
|---|---|
| Do once Only | Select this setting if you want to import the data from the source for only one time. In this case the import will start as soon as you click done in the last step of adding the dataset. |
| Repeat Task | This option will open the rest of the data schedule settings. Choose this option when you want the data to be imported frequently. |
| Starts | Choose the start date for importing the data. |
| Run Task Every | Choose the interval (month,week,day,hour) and the value. If you choose hour or day, then you can specify the days on which the schedule will run. |
| Repeat indefinately | Choose this if you want the schedule to run without an end date. In this case the schedule will stay active, until its deactivated again from the data load schedule page |
| Repeat Until | Set the end date of the schedule. |
| Full Refresh | This data load type deletes the whole dataset and reloads it again every time the load schedule runs.This is highly not recommended for large tables. Typically this is used for dimension tables that are not frequently updated and have less than 5000 records. |
| Partial Refresh | This data load type deletes and reloads dataset records up to a point back in time, every time the load schedule runs. This option requires a date field in your source table (e.g. Modified/Created timestamp). For example, you can setup the data load schedule to have a window of one month for a field such as “OrderDate”. This can be useful if the orders data change frequently within one month of the order date, but then the order is “closed” and will not be updated after the one month window period, so there will be no need to keep updating it. |
| Fetch new records | This is the recommended setting for data load jobs. This only brings new data records every time the schedule runs. This option requires a sequential field that can be a number or a date. For example, you can setup a load job to regularly import transactions based on a sequential field “TransactionID” or the date “TransactionTS” |
| Update Existing Records | At every schedule run, AnswerDock will refresh any record which has the selected date field updated. This date field is typically a Time Stamp that is used to mark the last time the record was updated in the source Table. |
| Disable Schedule | This saved the schedule details but disables it for the time being. |