Collecting data

Ellevio data

My utility company have an excellent online portal where I can see the hourly energy consumption for my home as well as the outside temperature. Unfortunately it is not the recorded temperature but the Weather services best guess. I know that the traffic authority have a weather station here so perhaps I can get exact data from them in the future.

Let’s analyze the data to see if there are any patterns that can help me in my desicion about upgrading the heat pump.

Parsing the data with ChatGPT

The data comes in two monthly Json files, one for consumption and one for temperature.

Temperature sample structure

{
    "data": [
        {
            "timeStamp": "2023-08-01T00:00:00+02:00",
            "temperature": 12.4
        }
    ]
}

Electricity sample structure

{
    "data": {
        "consumptions": [
            {
                "total": 1.833,
                "start": "2023-08-01T00:00:00+02:00",
            }
        ]
    }
}

Parsing the files with ChatGPT is quite straightforward. The only instructions that look a little like programming lingo is the way I ask it to parse the Json-files. It will make sense when you read it.

Preparing the data.

Every file contains hourly data for a month.

• 29 files named “ellevio_data_energy_202104.json” etc…
• 29 files named “ellevio_data_temperature_202104.json” etc…
• I zip them into two archives and upload them.

Instructing ChatGPT to parse the files

Initially, I used a dialog with the AI, but I’ve condensed it into a single instruction for easier readability.

I have two time series. One with consumption data and one with temperature data. You will parse them and merge them as follows.

In theLets start with parsing all the temperatures. I will give instructions in a home made format vaguely reminding of JSON path.

After parsing the two datasets you will merge them into an Excel-file

Parsing of dates:

• Parse the source ISO date into a format that Excel understands. YYYY-MM-DD HH:MM:SS
• Example: “2022-10-01T03:00:00+02:00” shall be transformed to “2022-10-01 03:00:00”
• Then convert it to the date-type

Parsing of temperature:

• Extract the objects of data array
• timeStamp parse as a date
• temperature is a number

Parsing of energy:

• Extract the objects of data/consumptions
• total is a number
• start parse as a date

Merging:

• Merge the data on date.
• Save start as column
• Save temperature as a column
• Save total as column

Filtering:

• Sort the data on time ascending
• Remove rows until you first find a non zero value in kW

Tests:

1. The first date in the series should be “2021-04-19 14:00:00”
2. The kW at “2022-08-16 15:00:00” should be “0”
3. The last row shall have the following values: 2023-08-31 23:00:00, 11.7, 2.934

Excel-file name: “trollebo-electricity-temperature-hourly”

Result

time	temperature	kW
2021-04-19 14:00:00	12.8	1.366
2021-04-19 15:00:00	13.1	1.367
2021-04-19 16:00:00	13.4	1.367
2021-04-19 17:00:00	13.3	1.81

Temperatur.nu data

Ellevio’s temperature data is an estimation, that frankly is really bad, especially for cold weather. Luckily we have Trafikverket’s (Swedish Transport Administration) weather station just outside the house and (Woohoo!) Temperatur.nu collects data from it! This data will be much more accurate. Before using privately remember to donate! For commercial use get a license. Temperatur.nu is an extremely impressive hobby project

Data collection.

Request: http://api.temperatur.nu/tnu_1.17.php?p=noppikoski&cli=unsigned&data&start=2021-04-20 00:00

This will download all hourly data from 2021-04-20 00:00 to current time.

I use Thunder Client to make the request. It’s a VS Code extension that makes it easy to make HTTP requests.

That’s all for now. We’ll examine the downloaded data on the next page.