Device Data (v2)

The v2 device data endpoints replace the single-formatter pipeline from v1 with a polymorphic formatter architecture where each standardCategoryId maps to a dedicated formatter class. v2 also expands the category lookup to include categoriesV2 and standardCategories (v1 only queries categories), introduces the CUSTOM aggregation type, and adds a side-by-side period comparison endpoint.

---

GET /api/user/deviceDataV2

GET /api/user/deviceDataV2?date1=09/11/2024&category=ID_SOURCE&type=DAY
Authorization: Bearer <access_token>

Parameters

Parameter	Type	Required	Default	Description
date1	string	Yes	—	DD/MM/YYYY — start date
date2	string	No	—	DD/MM/YYYY — end of range for comparison or CUSTOM type
category	string	Yes	ID_SOURCE	Category key resolved against the merged category lookup
divisionFactor	int	No	1000	Divisor applied to raw readings. Forced to 1 for GROUND_WATER_LEVEL regardless of this value
type	string	No	DAY	PatternTypeV2: HOUR, DAY, MONTH, or CUSTOM
unitId	string	No	—	When provided, filters the response to a single unit's data object
formattedOutput	boolean	No	true	When false, returns the raw data[] array from the service without running the formatter
v2	boolean	No	false	Currently applies only to energy category processing; has no effect on other categories

Validation

Condition	Error code	Message
date1 not provided	1403	—
category not provided	1403	—
category not found in merged category lookup	1401	Invalid category, `{category}` doesn't exist

format_args() — Argument Preparation

DeviceDataV2RouteHandler.format_args() runs before the data fetch. It performs date conversion, category resolution, and argument construction.

info

Date conversion uses the same shift-aware logic as v1:

1. days_offset = int(current_user['daysOffset'])
2. shift.startAt is read from current_user['industryData']['meta']
3. date1 is parsed as f'{date1}T{shift.startAt}' with format '%d/%m/%YT%H:%M:%S' → date1_local_with_shift
4. DateTimeUtil().convert_date_from_local_to_utc() converts to UTC
5. date1_utc_with_shift = utc_result - timedelta(days=days_offset)
6. Same for date2 if provided

Category lookup — merged across three sources:

all_categories = {
    **current_user['categories'],
    **current_user['categoriesV2'],
    **current_user['standardCategories']
}
selected_category = all_categories[args['category']]

note

v1 only searches current_user['categories']. v2 also checks categoriesV2 and standardCategories, allowing the same endpoint to serve both legacy and new category structures.

warning

divisionFactor override: If selected_category['standardCategoryId'] == 'GROUND_WATER_LEVEL', divisionFactor is forced to 1 regardless of the request parameter. Ground water level readings are stored in display units and must not be divided.

note

DeviceDataV2Args construction: The method builds a typed args object with date1 and date2 as Date(utc, local) pairs rather than raw strings, ensuring the correct coordinate set is available to each formatter.

CUSTOM type — date list generation: When type == 'CUSTOM', format_args() generates a dates[] list of Date objects covering every calendar day between date1 and date2 (inclusive). Each formatter iterates this list to produce per-day data slices.

Concurrent Data Fetch

Two futures run in parallel via ThreadPoolExecutor:

Future	Method	Result
1	get_last_data() → DeviceDataService.get_last_updated_time()	last_updated_time
2	get_devices_data() → DeviceDataService.get_data(...)	data

note

v2 does not fetch latest_data (Future 1 in v1) or quality week data (Future 4 in v1). The get_latest_data call and quality graph fetch are absent from this pipeline.

Formatter Selection — DeviceDataV2FormatterBuilder

After the data fetch, DeviceDataV2FormatterBuilder reads selected_category['standardCategoryId'] and instantiates the appropriate formatter:

standardCategoryId	Formatter class
FLOW_CATEGORY, sourceCategory	FlowDeviceDataFormatter
STOCK_CATEGORY	StockDeviceDataFormatter
ENERGY_CATEGORY	EnergyDeviceDataFormatter
ENERGY_MULTI	EnergyMultiDeviceDataFormatter
QUALITY_CATEGORY	QualityDeviceDataFormatter
GROUND_WATER_LEVEL	GWDeviceDataFormatter

note

Each formatter class extends DeviceDataV2FormatterTemplate and overrides category-specific methods for column names, unit conversions, aggregation logic, and response structure. This is in contrast to v1's single DeviceDataFormatter, which applies the same structure to all categories.

ALL_CATEGORIES_V2 Special Handling

info

If category starts with 'ALL_CATEGORIES_V2', the route enters aggregate mode:

1. The suffix of the category value is extracted as standardCategoryId
2. All entries in categoriesV2 whose standardCategoryId matches are collected
3. For each individual category, the formatter runs independently and its result is appended to a formatted_data['categories'] list
4. All sub-category values are summed into a synthetic TOTAL entry
5. The top-level subCategories key is removed from the response
6. The response shape becomes:

   {
     "categories": [ { "categoryId": "...", "data": {...} }, ... ],
     "total": { "value": 12345.67, "siUnit": "KL" }
   }

unitId Filter

tip

When unitId is provided, after formatting completes the route scans formatted_data['subCategories'] for a sub-category whose unit list contains the requested unitId. If found, the response is replaced with just that unit's data object rather than the full category tree. If not found, the full formatted response is returned unchanged.

formattedOutput=false

Bypasses the formatter entirely. The raw data[] array returned by DeviceDataService.get_data() is returned directly. Use this when the caller needs unprocessed readings for custom client-side aggregation.

GET /api/user/deviceDataV2?date1=09/11/2024&category=ID_SOURCE&type=DAY&formattedOutput=false
Authorization: Bearer <access_token>

{
  "data": [
    { "unitId": "U001", "timestamp": "2024-11-09T00:30:00Z", "value": 123456 },
    ...
  ]
}

Edge Cases

warning

Scenario	Behaviour
category matches GROUND_WATER_LEVEL	divisionFactor is forced to 1 before any processing; the request value is ignored
type == 'CUSTOM' and date2 not provided	dates[] list contains only date1; behaves identically to type == 'DAY' for a single day
category starts with 'ALL_CATEGORIES_V2' but no matching categoriesV2 entries	categories: [] and total: 0 returned; no error raised
unitId provided but not found in any sub-category	Full formatted category response returned; filter is a no-op
formattedOutput=false	Formatter is skipped; last_updated_time is not included in the response
v2=true on a non-energy category	Flag is read but has no effect; processing is identical to v2=false
divisionFactor not sent	Defaults to 1000; overridden to 1 for GROUND_WATER_LEVEL after default is applied

Status Codes

Code	Cause
200	Data returned successfully
401	Missing or invalid JWT
1401	category value not found in merged category lookup
1403	date1 or category missing

---

GET /api/user/deviceDataV2/compare

Runs two independent device data fetches for different date ranges and/or unit sets, then returns both results for side-by-side comparison.

GET /api/user/deviceDataV2/compare?dateA1=01/11/2024&dateB1=01/10/2024&ids1=U001&ids1=U002&ids2=U001&ids2=U002&category=ID_SOURCE&type=DAY&by=UNIT
Authorization: Bearer <access_token>

Parameters

Parameter	Type	Required	Default	Description
dateA1	string	Yes	—	DD/MM/YYYY — start of range A
dateA2	string	No	—	DD/MM/YYYY — end of range A
dateB1	string	Yes	—	DD/MM/YYYY — start of range B
dateB2	string	No	—	DD/MM/YYYY — end of range B
ids1	string (repeated)	Yes	—	Unit or sub-category IDs for period A. Repeat the parameter for multiple values
ids2	string (repeated)	Yes	—	Unit or sub-category IDs for period B
category	string	Yes	—	Category key — resolved against merged category lookup
divisionFactor	int	No	1000	Applied to both result sets
type	string	No	DAY	PatternTypeV2: HOUR, DAY, MONTH, CUSTOM
by	string	No	UNIT	UNIT or SUB_CATEGORY — controls how ids1/ids2 are interpreted

Execution Flow

info

by == UNIT:

A synthetic category is constructed:

REPORT_CATEGORY = {
    "subCategories": [
        { "id": "dummy", "displayName": "dummy", "units": ids1 }
    ]
}

This synthetic category is used for the period A fetch. The same structure is built for period B using ids2.

by == SUB_CATEGORY:

The base category is loaded from the merged lookup. Its subCategories list is filtered to only those entries whose id is in ids1 (for period A) or ids2 (for period B). The filtered category is passed to the data fetch.

Fetch execution:

DeviceDataV2RouteHandler().get() is called twice — once with (dateA range, filtered category for ids1) and once with (dateB range, filtered category for ids2). Both calls go through the full format_args() → concurrent fetch → formatter pipeline independently.

Response Shape

{
  "data1": {
    "subCategories": [...],
    "lastUpdatedTime": "...",
    "siUnit": "KL"
  },
  "data2": {
    "subCategories": [...],
    "lastUpdatedTime": "...",
    "siUnit": "KL"
  }
}

data1 corresponds to the dateA/ids1 result; data2 to the dateB/ids2 result. The response structure inside each is identical to a standard deviceDataV2 formatted response.

Edge Cases

warning

Scenario	Behaviour
ids1 and ids2 contain different unit sets	Each fetch runs against its own unit set independently; no cross-period unit alignment is enforced
by == SUB_CATEGORY and an ID in ids1 is not found in the base category	That sub-category is silently excluded from the filtered list
dateA range overlaps dateB range	Both fetches execute independently; overlapping data is returned in both data1 and data2

---

v1 vs v2 Comparison

Aspect	v1 (deviceData)	v2 (deviceDataV2)
Category lookup	current_user['categories'] only	Merged: categories + categoriesV2 + standardCategories
Formatter	Single DeviceDataFormatter for all categories	Polymorphic — DeviceDataV2FormatterBuilder selects class by standardCategoryId
type values	PatternType enum	PatternTypeV2 enum — adds CUSTOM
Latest data fetch	Future 1 runs when includeToday=True	Not fetched
Quality week fetch	Future 4 runs for HOUR + quality	Not fetched
Concurrent futures	4	2
GROUND_WATER_LEVEL	No special handling	divisionFactor forced to 1
Raw output mode	Not available	formattedOutput=false bypasses formatter
Period comparison	Not available	/compare endpoint
Aggregate across all categories	Not available	ALL_CATEGORIES_V2 prefix triggers aggregate mode

---

Status Code Reference

Code	Meaning
200	Success
401	Missing or invalid JWT
1401	category not found in merged category lookup
1403	Missing required parameter (date1, category, dateA1, dateB1, ids1, ids2)