I am 100% new to logging in an database at all and that's proberbly why there will maybe be some stupid questions here and i hope that it's ok. I would like to logg data from an Beckhoff plc controller into an DB that is placed on the same IPC as my PLC. The Beckhoff plc has a direct link function to both InfluxDB and to PostgreSQL that TimescaleDB is based on, so the connection will work fine. We would like to log data to time so we can go back and see what time certain things did happen and also make questions to the database based on time. I have been talking to different people and most of them recommend to use TimescaleDB so it would be great to hear the benefits between them and what you guys would recommend me to choose.
The data size we will log is pretty small. We will have an structure of data and that will contain like 10 INT registers so 20 bytes. We will make an log to the database evry 1 second on the quick machines and sometimes only one time each 20minutes, but this part i will control in my plc.
So putting data in the DB i belive will be pretty straight forward but then i have some toughts about what i would like to do and what is possible.
Is it possible to ask questions to the DB to give me the amount, higest value, lowest value, mean value the last 60 miuntes, or 24hours etc and then can the database retur these values based on the time frame i give the database in my question to it?
The resolution i log with that is controlled from the plc is only needed to be in that high for 7 days and after that i would like to "downstream / compress" the data. Is that possible in both these databases and is there any benefit in one of them? Maybe easier in one of them?
Is there in one of these two databases a possibility to not write to the HD / disk everytime my plc is putting data to it? Or it will write to the disk everytime automaticly? I did read about something called WAL, what is that or that will not use the RAM to store the data before it writes more data and not so often to the disk?
is there any big difference in setting up these two databases?
I proberly have more questions but these are the main functions that i need in the system.
Many thanks
Is it possible to ask questions to the DB to give me the amount, higest value, lowest value, mean value the last 60 miuntes, or 24hours etc and then can the database retur these values based on the time frame i give the database in my question to it?
Yes! You can use queries to make it. Consider the following table structure:
CREATE TABLE conditions (
time TIMESTAMPTZ NOT NULL,
device INTEGER NOT NULL,
temperature FLOAT NOT NULL
);
SELECT * FROM create_hypertable('conditions', 'time');
ALTER TABLE conditions SET (timescaledb.compress, timescaledb.compress_orderby='time');
The resolution i log with that is controlled from the plc is only needed to be in that high for 7 days and after that i would like to "downstream / compress" the data. Is that possible in both these databases and is there any benefit in one of them? Maybe easier in one of them?
You can create a continuous aggregates that is a fast method to keep your resumed data materialized.
CREATE MATERIALIZED VIEW conditions_hourly(time, device, low, high, average )
WITH (timescaledb.continuous) AS
SELECT time_bucket('1 hour', time) as time,
device,
min(temperature) as low,
max(temperature) as high,
AVG(temperature) as average
FROM conditions
GROUP BY 1,2;
And then you can add a retention policy for keeping only the last 7 days.
SELECT add_retention_policy('conditions', INTERVAL '7 day');
And add a continuous aggregates policy that will keep your view up to date every hour:
SELECT add_continuous_aggregate_policy('conditions_hourly',
start_offset => INTERVAL '1 day',
end_offset => INTERVAL '1 hour',
schedule_interval => INTERVAL '1 hour');
Is there in one of these two databases a possibility to not write to the HD / disk everytime my plc is putting data to it? Or it will write to the disk everytime automaticly? I did read about something called WAL, what is that or that will not use the RAM to store the data before it writes more data and not so often to the disk?
In Postgresql you can use async commits: https://www.postgresql.org/docs/current/wal-async-commit.html