It is the process of updating the data in a BI
Note: in many topics, especially when we talk about loads or structure, let´s talk about "Origin", what is Origin? This is where the information that will be stored in BIMachine will come from.
In BIMachine we have 2 types of INCREMENTAL and TOTAL loads. Both can be done IMMEDIATELY, it means, runs when available in the queue, or with a schedule stipulating and update period. The image below shows an example of a daily charge being made at 8:00 am daily. The period type are:
- INTERVAL: starting from a start and end time in a time interval to define (that value, is minute if you write 60 will be hourly), and defining the days of the week that you want.
- DIARY: Setting the days of week that you want and the schedule
- WEEK: Setting one of the days of the week and the schedule.
- MONTHLY: Defining the day to the month and the schedule. Be careful if the month has not 31 days, it won´t update.
Updated without removing ALL data, it will be deleted if you define a PERIODICITY over a time dimension (current year, previous month, etc …) See in the image below, an example of daily load, removing over a time dimension called "DATE" and using the "CURRENT DAY" periodicity. The periodicities are:
- Dynamic: you define a periodicity to remove (Current day, previous year, last 3 months, etc, …) BUT CAUTION if the source comes from information beyond this periodicity, the data may be duplicated.
- Data: will pick up the smallest data coming from the sources and it will remove the data structure with that track.
- Fixed: you define a start and end track to remove. But TAKE CARE is given to the source of the information provided, it is possible to duplicate.
For example, if running and total and the source brings data for the current month and BIMachine you have more data from the current month, after updating, you will only have the data for the current month.
Recommendation: do not use this charge during office hours and perform several times. This type of cargo is recommended at dawn or on weekends.
Type structure
In BIMachine we have 2 types of structures BIGTABLE and NORMAL. In BIGTABLE, loads take longer and objects display faster, in NORMAL, faster loads and objects display slower. This type of definition goes according to the size of the information that comes from the source, that is if the size is little we recommend the NORMAL type, if the size is very large we recommend the BIGTABLE. See the size of a structure below to see if it is very large or not.
See below how to create a schedule for each type of source we have in BIMachine:
- SQL: Updating SQL structure
- CSV: Updating CSV structure
- CSV (Remote): Updating CSV (Remote) structure
- uMov.me: Updating uMov.me structure
- BIMachine: Updating BIMachine structure
- Facebook: Updating Facebook data
- PipeDrive: Updating Pipedrive data
- Piperun: Updating PipeRun data