Water Balance

The Water Balance feature models an industry's entire water network as a directed graph — physical meter nodes (real devices) connected by edges to virtual aggregate nodes (computed totals). This graph is saved per-industry and then queried with a date/time window to populate each node and edge with actual flow values.

info

The graph configuration is stored in the additional_feature Cosmos DB container under document ID WATERBALANCE_{industryId} with type: "waterBalance".

Concepts

Node Types

Node type	Source	Value
Physical node	A real metered unit (`unitId` in `unitsMapping`)	Raw device reading fetched from Cosmos DB, divided by `divisionFactor`
Virtual node	A user-defined aggregate	Sum of all incoming physical nodes, or the result of a custom `calculations` formula
Virtual reference node	A virtual node with `editmeta.referencedNodeId` set	Resolved at runtime to the referenced physical node's live value — rendered at the reference node's canvas position
Disabled node	Any node present in the saved graph but not connected by any edge	Carried through and returned in the response with `addDisabledNode=True` — value is `null`

Graph Structure

Each save replaces the entire graph document. The GraphManager maintains multiple disconnected sub-graphs (connected components) in memory — when an edge is added between two previously separate components, they are merged into one. This means an industry can have multiple independent water networks rendered on the same canvas.

GET /api/user/waterBalance/graphEditor

Returns the saved graph layout for the graph editor UI. No date parameters — this is the raw topology, not populated with values.

GET /api/user/waterBalance/graphEditor
Authorization: Bearer <access_token>

Execution flow

WaterBalanceService.__init__() runs load_waterbalance_document() which queries:
```
SELECT * FROM c WHERE c.industryId = @industryId AND c.type = "waterBalance"
```
from additional_feature container. If no document exists, all internal lists (edges, physicalNodesMapping, virtualNodesMapping) are empty.
get_graph() is called:
- Iterates unitsMapping (all units for the industry from current_user) and merges any saved physicalNodesMapping metadata (canvas position, editmeta, rotation) into each unit's data
- If no physicalNodesMapping exists yet (first-time setup), uses bare unitsMapping values — all units appear as unpositioned nodes
- Calls GraphManager.build_graph_from_db_data(edges, physical_nodes, virtual_nodes) which rebuilds the in-memory graph from the DB-format data
- Returns edges, nodes, and metadata (pages config)

WaterBalanceDataFormatter.graph_editor_format_data() shapes each node into:

{
  "id": "U001",
  "data": { "label": "Main Inlet", "rotation": 0 },
  "position": { "x": 120, "y": 340 },
  "nodeType": "physicalNode",
  "type": "OVAL",
  "meta": { "graphTag": "physical", "referencedNodeId": null }
}

And each edge into:

{
  "id": "edge-U001-VTOTAL",
  "source": "U001",
  "target": "VTOTAL",
  "type": "custom",
  "sourceHandle": "right",
  "targetHandle": "left",
  "data": { "controlPoints": [...] }
}

metadata (pages config from the saved document) is passed through to the response as-is.

Response shape

{
  "nodes": [
    {
      "id": "U001",
      "data": { "label": "Main Inlet", "rotation": 0 },
      "position": { "x": 120, "y": 340 },
      "nodeType": "physicalNode",
      "type": "OVAL",
      "meta": { "graphTag": "physical" }
    },
    {
      "id": "VTOTAL",
      "data": { "label": "Total Consumption", "rotation": 0 },
      "position": { "x": 500, "y": 340 },
      "nodeType": "virtualNode",
      "type": "RECTANGLE",
      "meta": { "graphTag": "physical" }
    }
  ],
  "edges": [
    {
      "id": "edge-U001-VTOTAL",
      "source": "U001",
      "target": "VTOTAL",
      "type": "custom",
      "sourceHandle": "right",
      "targetHandle": "left",
      "data": { "controlPoints": [] }
    }
  ],
  "metadata": {
    "pages": [
      { "id": "physical", "displayName": "Water Flow", "order": 0, "icon": "droplet" }
    ]
  }
}

note

If no graph has been saved yet, nodes contains all industry units (from unitsMapping) with random x/y positions in the [-100, 100] range, and edges is empty.

Status	Cause
`200`	Graph layout returned
`401`	Missing or invalid JWT

POST /api/user/waterBalance/graphEditor

Saves the entire graph configuration. Full replace — the previous document is overwritten completely.

POST /api/user/waterBalance/graphEditor
Authorization: Bearer <access_token>
Content-Type: application/json

Request body

{
  "edges": [
    {
      "id": "edge-U001-VTOTAL",
      "source": "U001",
      "target": "VTOTAL",
      "type": "custom",
      "sourceHandle": "right",
      "targetHandle": "left",
      "data": { "controlPoints": [{ "x": 300, "y": 340 }] }
    }
  ],
  "nodes": [
    {
      "id": "U001",
      "nodeType": "physicalNode",
      "data": { "label": "Main Inlet", "rotation": 0 },
      "position": { "x": 120, "y": 340 },
      "meta": { "graphTag": "physical" }
    },
    {
      "id": "VTOTAL",
      "nodeType": "virtualNode",
      "data": { "label": "Total Consumption", "rotation": 0 },
      "position": { "x": 500, "y": 340 },
      "type": "RECTANGLE",
      "meta": {
        "graphTag": "physical",
        "calculations": "{U001} + {U002}",
        "units": ["U001", "U002"]
      }
    }
  ],
  "metadata": {
    "pages": [
      { "id": "physical", "displayName": "Water Flow", "order": 0 }
    ]
  }
}

Execution flow

GraphManager.build_graph_from_frontend_data(edges, nodes) is called:
- Deduplicates edges and nodes by id (last-write wins)
- Builds physical and virtual node objects from the payload
- Processes edges one by one, merging connected components: if source and target belong to different sub-graphs they are merged into one; if neither belongs to any sub-graph yet, a new sub-graph is created
- Nodes that end up with no edge connections are tracked as disabledNodesMap
GraphManager.convert_to_db_data(addDisabledNode=True) serialises back to DB format, stripping frontend-only fields. Only these physical node fields are persisted:
```
{ "id": "U001", "position": { "x": 120, "y": 340 }, "nodeType": "physicalNode", "editmeta": {} }
```
Unit display names and device config are not stored — they are always read live from unitsMapping at query time.
DatabaseSupporter.create_additional_feature_doc(...) upserts the document with ID WATERBALANCE_{industryId}.

warning

customSummary and summaryGraph fields from the existing document are not preserved — if the frontend doesn't include them in the POST body, they are lost. The metadata field (pages config) is taken directly from args.get("metadata", {}).

Response

{ "status": "success", "message": "data saved" }

Status	Cause
`200`	Graph saved
`401`	Missing or invalid JWT

GET /api/user/waterBalance/data

Returns the water balance graph with all physical nodes populated with real device readings for the given date/time window, virtual node totals computed, and results organised into pages.

GET /api/user/waterBalance/data?date1=09/11/2024&type=DATE
Authorization: Bearer <access_token>

Parameters

Parameter	Required	Default	Description
`date1`	Yes	—	`DD/MM/YYYY` — the date to query. `1403` if missing
`date2`	No	—	`DD/MM/YYYY` — end of range (for multi-day views)
`divisionFactor`	No	`1000`	Raw value divisor applied before display
`type`	No	`HOUR`	`HOUR` — hourly flow data within the day; `DATE` — daily aggregated totals

Execution flow

1. Service initialisation

WaterBalanceService.__init__() loads current_user context and calls load_waterbalance_document() to hydrate edges, physical/virtual node mappings, pages config, and summary graph from the additional_feature document.

2. Data fetching — `get_waterbalance_graph_data(args)`

The route sets args["summaryPage"] = True before calling this method.

For type=HOUR (default):

A single fetch_device_data() call is made with category=None (all categories)
DeviceDataService.get_data(date1_utc, date2_utc) queries Cosmos DB for hourly-bucketed readings

For type=DATE:

Two parallel fetches are made — one for sourceCategory units and one for stockCategory units
sourceCategory fetch: DeviceDataService.get_data(date1_local, date2_local) — daily totals
stockCategory fetch has special logic:
- If date1 is in the current month: fetch normally (live closing stock)
- If date1 is a past month: rewind date1 to the last day of that month to get the month-end closing stock value
Results from both fetches are concatenated into a single data1 list

Today detection — is_date_today(date1, timezone):

If date1 matches today in the industry's timezone, an additional concurrent get_latest_data() call fetches the most recent raw sensor reading for each unit. This "latest" value is merged into the historical aggregated data via update_latest_data_to_old_data() — ensuring that the current period shows the freshest possible reading rather than a potentially stale last-bucket value.

All three calls (get_devices_data, get_last_data, get_latest_data) run concurrently via ThreadPoolExecutor.

fetch_device_data() also:

Filters current_user['categories'] to only sourceCategory and stockCategory before calling DeviceDataRouteHandler().get_date1_and_date2() for UTC conversion
Appends all virtual units (units with standardCategoryId == "virtualCategory") to sourceCategory's first subcategory unit list before fetching

3. Graph loading — `GraphManager.load_waterbalance_data()`

Builds the in-memory graph with values attached:

Physical nodes: each node gets processed_data_obj (the fetched device reading for its unitId), last_updated_time_obj, siUnit from CachedData.standardCategoriesMap, and isRecycledWaterStock flag (set if the unit is in standardCategoriesView['ID_RECYCLED_WATER_STOCK'])
Virtual reference nodes: the referencedNodeId chain is resolved (up to depth 10) — if the final resolved node is a physical unit, the reference node is converted to a physical node type at render time, using the referenced unit's live data but the reference node's canvas position and label
Virtual nodes (non-reference): rendered as aggregate nodes; their values are computed in the next step
Edges: rebuilt with controlPoints, sourceHandle, targetHandle, and args context attached

4. Graph processing — `get_graph_data(process=True)`

node.process() is called on each node — physical nodes compute their display value from processed_data_obj; virtual nodes sum their incoming physical node values
Consumption tagging: if is_today=True, nodes with meta.isConsumption=True AND online=True are collected into isConsumptionNodes. Any edge whose target is in isConsumptionNodes gets isConsumption=True tagged on the edge — used by the frontend to apply consumption-specific styling

5. Page grouping

The pages list from metadata is separated into:

Main diagrams — pages without _summary in their ID
Summary diagrams — pages with _summary suffix (or "summary" for the physical page)

For each main diagram page, get_graph_data(graphTag=page_id) filters nodes/edges by editmeta.graphTag == page_id. The companion summary diagram (if it exists) is fetched the same way using graphTag={page_id}_summary.

get_summary() computes the pageSummary strip using the page's customSummary config:

Legacy mode (standardCategoriesView-based): sums nodes in ID_SOURCE, ID_CONSUMPTION, ID_RAW_WATER_STOCK categories. The stock label changes between "Total Stock Available" (today) and "Closing Stock" (past dates)
Custom formula mode (customSummary with calculations): evaluates arithmetic expressions like {nodeId1} + {nodeId2} against current node values. Missing node IDs default to 0. Result is multiplied by 1000 before formatting

6. Formatting — `WaterBalanceDataFormatter.format_data_pages()`

For each page:

Physical node values: round(raw_value / 1000, 2) with siUnit appended as a string (e.g. "1234.56 KL")
Virtual node values: same division
WATER_BALANCE type nodes in summary diagrams: not divided — their value is already in the display unit
All other summary nodes: divided by 1000
pageSummary values: divided by 1000 and rounded to 2 decimal places
Edge isConsumption flag is passed through to the response

Response shape

{
  "pages": {
    "physical": {
      "id": "physical",
      "displayName": "Water Flow",
      "order": 0,
      "icon": "droplet",
      "diagram": {
        "nodes": [
          {
            "id": "U001",
            "unitId": "U001",
            "data": {
              "label": "Main Inlet",
              "value": "1234.56 KL",
              "lastUpdatedTime": "2024-11-09T10:30:00Z",
              "online": true,
              "meta": { "isConsumption": false, "limitReached": false },
              "subCategoryId": "FLOW_CONSUMPTION",
              "rotation": 0
            },
            "type": "OVAL",
            "position": { "x": 120, "y": 340 },
            "nodeType": "physicalNode",
            "categoryTypeId": "sourceCategory"
          },
          {
            "id": "VTOTAL",
            "unitId": "VTOTAL",
            "data": {
              "label": "Total Consumption",
              "value": "2500.00 KL",
              "online": true,
              "meta": {}
            },
            "type": "RECTANGLE",
            "position": { "x": 500, "y": 340 },
            "nodeType": "virtualNode",
            "categoryTypeId": "custome"
          }
        ],
        "edges": [
          {
            "id": "edge-U001-VTOTAL",
            "source": "U001",
            "target": "VTOTAL",
            "type": "custom",
            "data": { "controlPoints": [], "isConsumption": true }
          }
        ]
      },
      "pageSummary": [
        { "displayName": "Total Source", "value": 3500.00, "siUnit": "KL", "nodes": [...] },
        { "displayName": "Total Consumption", "value": 2500.00, "siUnit": "KL", "nodes": [...] },
        { "displayName": "Closing Stock", "value": 1000.00, "siUnit": "KL", "nodes": [...] }
      ],
      "summary": null,
      "siUnit": "KL"
    }
  }
}

Edge cases

Scenario	Behaviour
No graph saved for the industry	All fields empty — `nodes: []`, `edges: []`, `pages: {}`
Physical node has no device data for the date	`value: null` in node data; edge `isConsumption` not set for null-value nodes
`date1` is today	`get_latest_data()` runs concurrently and its value overwrites the last historical bucket for each unit
`date1` is a past month with `type=DATE`	Stock nodes query the last day of that month, not `date1`
Virtual reference node's target is missing	Reference chain resolution stops at depth 10 and returns the last found node
Circular dependency in `calculations` formula	Detected via dependency graph; circular nodes are set to `0` and a warning is printed
Node in `calculations` formula references an unknown node ID	`DefaultDict.__missing__` returns `0.0` — formula evaluates without error
Page has no companion `_summary` page	`formatted_page["summary"]` is `null`
`isConsumption` consumption tagging	Only applies when `is_today=True`; past-date queries never tag edges as consumption

Status	Cause
`200`	Page data returned
`401`	Missing or invalid JWT
`1403`	`date1` missing

Storage Document

The full Cosmos DB document written by POST /graphEditor:

{
  "id": "WATERBALANCE_IND001",
  "industryId": "IND001",
  "type": "waterBalance",
  "edges": [
    {
      "id": "edge-U001-VTOTAL",
      "source": "U001",
      "target": "VTOTAL",
      "type": "custom",
      "sourceHandle": "right",
      "targetHandle": "left",
      "controlPoints": [{ "x": 300, "y": 340 }],
      "markerEnd": { "type": "arrowclosed" },
      "style": {}
    }
  ],
  "nodes": {
    "physicalNodes": [
      {
        "id": "U001",
        "position": { "x": 120, "y": 340 },
        "nodeType": "physicalNode",
        "editmeta": { "graphTag": "physical" }
      }
    ],
    "virtualNodes": [
      {
        "id": "VTOTAL",
        "label": "Total Consumption",
        "position": { "x": 500, "y": 340 },
        "nodeType": "virtualNode",
        "type": "RECTANGLE",
        "editmeta": {
          "graphTag": "physical",
          "units": ["U001", "U002"],
          "calculations": "{U001} + {U002}",
          "backgroundColor": "#e8f4f8"
        }
      }
    ]
  },
  "metadata": {
    "pages": [
      { "id": "physical", "displayName": "Water Flow", "order": 0, "icon": "droplet" }
    ]
  },
  "customSummary": {},
  "summaryGraph": {}
}

Key field notes

nodes.physicalNodes stores only layout metadata (position, editmeta, nodeType) — device config is always read live from unitsMapping
nodes.virtualNodes stores the full virtual node definition including label, calculations, and units
edges store full topology including handle positions and control points for curved edges
metadata.pages drives the page-tab UI and the graphTag filtering in get_graph_data()
customSummary / summaryGraph are legacy fields; active summary config lives in each virtual node's editmeta

Status Code Reference

Code	Meaning
`200`	Success
`401`	Missing or invalid JWT
`1403`	Missing required parameter (`date1`)

Concepts​

Node Types​

Graph Structure​

Pages​

GET /api/user/waterBalance/graphEditor​

Execution flow​

Response shape​

POST /api/user/waterBalance/graphEditor​

Request body​

Execution flow​

Response​

GET /api/user/waterBalance/data​

Parameters​

Execution flow​

1. Service initialisation​

2. Data fetching — get_waterbalance_graph_data(args)​

3. Graph loading — GraphManager.load_waterbalance_data()​

4. Graph processing — get_graph_data(process=True)​

5. Page grouping​

6. Formatting — WaterBalanceDataFormatter.format_data_pages()​

Response shape​

Edge cases​

Storage Document​

Status Code Reference​

Concepts

Node Types

Graph Structure

Pages

GET /api/user/waterBalance/graphEditor

Execution flow

Response shape

POST /api/user/waterBalance/graphEditor

Request body

Execution flow

Response

GET /api/user/waterBalance/data

Parameters

Execution flow

1. Service initialisation

2. Data fetching — `get_waterbalance_graph_data(args)`

3. Graph loading — `GraphManager.load_waterbalance_data()`

4. Graph processing — `get_graph_data(process=True)`

5. Page grouping

6. Formatting — `WaterBalanceDataFormatter.format_data_pages()`

Response shape

Edge cases

Storage Document

Status Code Reference