Skip to main content

Ingestion API User Guide

User guide to learn how to access TWAICE API for data ingestion.

Ece Aras avatar
Written by Ece Aras
Updated over 12 months ago

Introduction

Welcome to the TWAICE Ingestion API documentation. TWAICE Ingestion API allows you to integrate your battery data with our advanced analytics platform, providing insights into battery performance, health, and safety. Our Ingestion API enables seamless data ingestion to optimize your battery operations.

This document aims to guide through how to access TWAICE API for data ingestion.

Introduction to TWAICE API:

  • The TWAICE Ingestion API follows REST architecture principles.

  • All API requests are accepted only utilizing HTTPS.

  • The TWAICE Ingestion API accepts and returns JSON format for both requests and responses. The Content-Type request header field should be set to application/json during interaction.

Authentication

  • Authentication is done via JWT Tokens as the TWAICE API Authorization Key.

  • To obtain a key token, use the authentication API endpoint with your email and password as credentials.

  • Tokens can only be obtained by registered customers with valid credentials. Authentication credentials should have been created for you, otherwise reach out to your TWAICE contact.

  • Once you obtain the idToken from the endpoint, you can pass your Key as Bearer token in an Authorization Header.

  • Lifespan of the API auth token is 1 hour.

> Authorization: Bearer JWT_TOKEN

How to Authenticate

Authenticate the user to get a token to use for ingesting data.

email (required)

string

password (required)

string

Request sample

POST /auth/authenticate

API endpoint: https://api.twaice.ai/auth/authenticate

Payload

Content type: application/json

{
  "email": "johnDoe123@gmail.com",
  "password": "d4f089b1e7a6"
}

Responses

HTTP 201 "Created": Login with provided credentials is successful

Response Body Schema: application/json

idToken

string

JWT Token used for API request authentication & authorization

Response Sample:

{
  "idToken": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiI1ZGYy"
}

HTTP 400 "Bad Request": Validation Error for the request input

Response Body Schema: application/json

status (required)

string

Enum: "success", "error"

data (required)

object

message (required)

string

Error message if status is error

Response Sample: 

{
  "status": "error",
  "data": { 
     "statusCode": 400, 
     "message": [ 
       "email should not be empty", 
       "Either email or username should be provided." 
      ], 
    "error": "Bad Request"
  },
  "message": "Bad Request Exception"
}

HTTP 401 "Unauthorized": Invalid login credentials

Response Body Schema: application/json

status (required)

string

Enum: "success", "error"

data (required)

object

message (required)

string

Error message if status is error

Response Sample:

{
  "status": "error",
  "data": {
     "statusCode": 401, 
     "message": "Incorrect username or password.", 
     "error": "Unauthorized"
  },
  "message": "Incorrect username or password."
}

Ingestion

Ingest raw customer measurements with TWAICE API.

Authorization

  • HTTP Authorization Scheme: bearer

  • Header: Authorization: Bearer idToken - value from the authentication request (see How to Authenticate section)

  • Bearer format: JWT

POST /ingestion/customers/{customer}/systems/{system}/measurements

API Endpoint: https://api.twaice.ai/ingestion/customers/{customer}/systems/{system}/measurements

Path Parameters

customer (required)

string [ 1 .. 32 ] characters ^[a-z0-9-]*$

system (required)

string [ 1 .. 32 ] characters ^[a-z0-9-]*$

  • Customer and System have to be between 1 and 32 characters in length, containing letters, numbers and dashes.

  • They will be provided by TWAICE to the user.

Request Body Schema

Array ([ 1 .. 1000 ] items): Accepts up to 1000 measurement objects per request (limit can be modified in need, should be discussed with TWAICE)

sensor (required)

string [ 1 .. 200 ] characters ^[^/]*$

Customer provides sensor identifier unique per system of up to 200 characters length, containing letters, numbers, special characters excluding "forward slashes ("/").

timestamp (required)

number [946684800 .. 4102444800]

Unix timestamp in seconds with optional Microseconds as fraction. The unix timestamp is the number of seconds between a particular date and the Unix Epoch on January 1st, 1970 at UTC.

  • Must be after 2000-01-01 00:00:00 GMT+0000

  • Must be before 2100-01-01 00:00:00 GMT+0000

Example:

1642770062.123456 == 2022-01-21T14:01:02.123456Z, with the fraction representing 123.456 milliseconds or 123456 microseconds respectively

value (required)

number

Value of a sensor at the given Timestamp.

Accepts integers or float64 (double)

Request Sample:

Payload

[
  {
    "sensor": "Sensor XYZ",
    "timestamp": 1642770062.123456,
    "value": 12.345678
  }
]

Responses

HTTP 200: Successful Response

Response Body Schema: application/json

numberOfRecordsIngested (required)

integer

>= 0

On successful 200 response, code returns the number of measurements ingested into the TWAICE platform through the respective POST request.

Response Sample:

{
  "numberOfRecordsIngested": 1
}

HTTP 404: Not Found

Returns in the case of if there is invalid path parameters.

Response Body Schema: application/json

type

string

Default: "about:blank"

A URI reference that uniquely identifies the problem type only in the context of the provided API. Opposed to the specification in RFC-7807, it is neither recommended to be dereferenceable and point to a human-readable documentation nor globally unique for the problem type.

title

string

A short summary of the problem type. Written in English and readable for engineers, usually not suited for non technical stakeholders and not localized.

status

integer [100 .. 600)

Default: 500

The HTTP status code generated by the origin server for this occurrence of the problem.

detail

string

A human readable explanation specific to this occurrence of the problem that is helpful to locate the problem and give advice on how to proceed. Written in English and readable for engineers, usually not suited for non technical stakeholders and not localized.

timestamp

number [946684800 .. 4102444800]

Unix timestamp in seconds with optional Microseconds as fraction. The unix timestamp is the number of seconds between a particular date and the Unix Epoch on January 1st, 1970 at UTC.

  • Must be after 2000-01-01 00:00:00 GMT+0000

  • Must be before 2100-01-01 00:00:00 GMT+0000

Example:

1642770062.123456 == 2022-01-21T14:01:02.123456Z, with the fraction representing 123.456 milliseconds or 123456 microseconds respectively

Response Sample:

{
  "type": "about:blank",
  "title": "HTTPException",
  "status": 404,
  "detail": "Not Found",
  "timestamp": 1642770062.123456
}

422: Unprocessable Entity

Returns in the case of request contains invalid data.

Response Body Schema: application/json

type

string

Default: "about:blank"

A URI reference that uniquely identifies the problem type only in the context of the provided API. Opposed to the specification in RFC-7807, it is neither recommended to be dereferenceable and point to a human-readable documentation nor globally unique for the problem type.

title

string

A short summary of the problem type. Written in English and readable for engineers, usually not suited for non technical stakeholders and not localized.

status

integer [100 .. 600)

Default: 500

The HTTP status code generated by the origin server for this occurrence of the problem.

detail

string

A human readable explanation specific to this occurrence of the problem that is helpful to locate the problem and give advice on how to proceed. Written in English and readable for engineers, usually not suited for non technical stakeholders and not localized.

instance

string

A URI reference that identifies the specific occurrence of the problem, e.g. by adding a fragment identifier or sub-path to the problem type. May be used to locate the root of this problem in the source code.

timestamp

number [946684800 .. 4102444800]

Unix timestamp in seconds with optional Microseconds as fraction. The unix timestamp is the number of seconds between a particular date and the Unix Epoch on January 1st, 1970 at UTC.

  • Must be after 2000-01-01 00:00:00 GMT+0000

  • Must be before 2100-01-01 00:00:00 GMT+0000

Example:

1642770062.123456 == 2022-01-21T14:01:02.123456Z, with the fraction representing 123.456 milliseconds or 123456 microseconds respectively

errors

Array of objects

A list of error objects with detailed information about each of the subproblem, which occurred in API call

Response Sample:

{
  "type": "https://docs.twaice.ai/#section/422:-Unprocessable-Entity",
  "title": "Unprocessable Entity",
  "status": 422,
  "detail": "Request contains invalid data, more information about the specific problem could be found in errors field",
  "instance": "/system/11223344/sensor/55667788",
  "timestamp": 1642770062.123456,
  "errors": [
    {
      "title": "value_error.missing",
      "detail": "field required",
      "instance": "/body/timestamp"
    }
  ]
}

500: Internal Server Error

Returns in the case of error or failure in the backend service.

Test Environment

TWAICE provides a sandbox environment where you can test your API integration without affecting live data. 

> Authorization: Bearer JWT_TOKEN

Path Parameters

Same applied in Path Parameters under Ingestion. They will be provided by TWAICE to the user.

Request Body Schema

Same applied in Request Body Schema under Ingestion.

  • Sensor IDs need to be defined in advance during the system configuration stage

  • For the sandbox system, any sensor_bk can be used, as the data will not be passed through the complete pipeline

Request Sample

[
  { 
    "sensor": " string 1.1.1-string voltage", 
    "timestamp": 1642770062.123, 
    "value": 401.2
   },
   { 
    "sensor": " string 1.1.1-string voltage ", 
    "timestamp": 1642770062.123, 
    "value": 403.1
   }
]

Frequently Asked Questions

Are there any other options for submitting to the API besides a RESTful service?

  • Our API currently only supports the above mentioned RESTful methods.

How much historical data will be required when onboarding a site? What format is expected?

  • 1 month would be optimal. Via API or in a file based format standardized by TWAICE on request.

At what resolution will historical data be required? Is this the same tag list as for near real-time data?

  • Same tag list is applied. Resolution should be the same in best case (2 second if available, up to 60 seconds)

Is it possible to send large data for initial site onboarding to S3 rather than the API?

  • Yes, via SFTP or via S3 access keys is possible.

Can the storages send the data lower resolution then 2 second resolution (e.g. 30 seconds)?

  • 2 to 60 seconds resolution is accepted, however lower resolution then 2 second might affect accuracy of some KPIs.

How often should data be transmitted? As soon as it’s available (i.e., near real-time) or can be batches? What is the impact of delayed data?

  • For API best would be to fill the 1000 maximum batch size.

  • Frequency and rate-limits will be discussed during onboarding, however delayed data transmission might lead to delayed KPI calculations or notifications/ alerting.

Must data always be delivered in chronological order?

  • Not needed, sorted based on timestamp. However, minimum delay is expected as mentioned in previous answer.

Can we send all data in UTC?

  • All timestamps sent as defined in the ingestion schema are expected to be UNIX timestamps, which by definition are UTC based.

How would TWAICE like to be notified of failed data transmissions?

  • We can detect it as “missing data” and notify. Customer can just resend the data or if longer outages happen, send an email at support@twaice.com

How should a storage handle resending data that did not land via the API? How often can we retry and what is the max number of retries we would like to configure?

  • There is no maximum retry from our side. If data comes in too late we might not be able to consider them for KPI calculation or alerting feature.

How would our system play “catch-up” if the TWAICE API becomes unavailable for some reason?

  • We inform the customer and fix it as soon as possible.

Does TWAICE have anomaly detection to prune poor quality data?

  • We have threshold-based cleansing rules, need configuration for each storage and customer based on needs.

Related Articles
TWAICE Simulation Model - Matlab Simulink: How to Get Started
Ingestion via TWAICE SFTP Server
BESS Sensor Requirements & Model Components
Did this answer your question?