To configure the Pull Stack for efficient and cost-effective data ingestion, TWAICE kindly requests that the following information is provided. This will help us align the pull configuration with your system capabilities and avoid overloading your infrastructure.
This questionnaire must be answered prior to any configuration or setup efforts conducted by TWAICE.
In this article, the term "sensor" is equivalent to "tag."
Data Pull Configuration
Sensor Batch Size
What should be the size of the sensor / tag batch per pull? Is there a maximum number of sensors / tags that can be pulled simultaneously?
Time Range per Pull Batch
What is the desired time range for each pull batch?
Data Delay Interval
How much delay should we include between the current time and the
pulling interval to account for any delayed incoming data?
Parallel Pulls
What is the maximum number of parallel pulls that should be allowed (to
balance speed and system load)?
Retention Period
For how long should data be retained in case a pulling attempt fails?
Sensor Grouping (Optional)
Should sensors / tags be grouped per component (e.g., pulling data only from one PCS per batch)? If yes, please specify the grouping rules.
Rate Limits
What are your system’s rate limits (e.g., requests per second, requests per
day, maximum number of sensors / tags per pull)?
Parallel Access
How many parallel workers can pull data from your data source using the
provided credentials?
Data Delay Handling
What is the maximum expected delay time for data in case of a data gap?
What is the typical data delay time under normal conditions?
When can we usually expect the latest measurement to be available in the
data source?
Data Storage
What type of data store are you using (e.g., SQL database, time-series database, data lake, API service)?
Documentation & Access
Do you have up-to-date technical documentation for the data source?
Can you provide a sample Python snippet demonstrating how to connect (Authentication) to and query the data?
Data Response Format
In what format is the data returned (e.g., JSON, CSV, XML)?
Which fields are included in the response (please provide a field list or schema)?
Downloadable Version
For convenience, these questions are also provided as a Microsoft Word document.
If a pull stack connection will be implemented, please provide a copy of your responses to your Customer Success contact or to support@twaice.com.