Roadmap Proposal

[daadu]

Last Updated - 17 Aug 2021

django-timescaledb

django-timescaledb aims to achieve following objectives

• feature rich integration of TimescaleDB with Django
• high-quality module - proper testing and documentation
• keeping the module active with future releases of TimescaleDB and Django
• clearly communicate - version supports for Django and TimescaleDB - along with LTS, deprecation/upgrade notes

Improvements/Feature Requests

• time as primary_key #22 Why not allow making "partition key" as primary key? [M.1]
• Why does the partition key have to be named "time"? [M.1]
• data retention policy [M.2]
• Compression policies [M.2]
• FR: Aggregate View #12 Continuously aggregated views [M.2]
• additional partitioning key [M.2]

Milestone 1 [Basic usage]

• Create hypertable (for empty or non-empty tables) with single "time" based partition
• proper model.Manager that can be used to run queries
• proper testing framework
• proper documentation
• all features in this step should work seamlessly with Django's makemigrations and migrate commands, without user having to run single query himself
• APIs
  • DB backends (postgres and postgis) - TimescaleSchemaEditor
  • models.Model - TimescaleModel (abstract)
  • models.Manager - TimescaleManager (time_bucket, time_bucket_gapfill, histogram)
  • models.Field - TimescaleDateTimeField
  • models.Func - Interval, TimeBucket, TimeBucketGapFill
  • models.Aggregate - Histogram, Last, First

Milestone 2 [Advance features with manual management]

• have migrations operations for adding/removing/altering advanced features of TSDB, this has to be dropped in by the user in the migrations files manually
• support for "AggregateViews" - not sure how to implement it?
• additional partitioning key - not sure how to implement it?
• additional functions and/or queryset methods - "Drop chunks", "Percentile approximation", "Time-weighted average functions"
• new APIs
  • data retention policy - AddRetentionPolicy, AlterRetentionPolicy, RemoveRetentionPolicy
  • compression - AlterTableCompression, AddCompressionPolicy, RemoveCompressionPolicy

Milestone 3 [Advance features with automatic management]

• Have a parallel "migration system" like Django's
  • Model.TSMeta class - since Model.Meta class in Django is restrictive (cannot add new attributes, etc), need to have a parallel TSMeta class for storing "TSDB related meta" info (policies, etc) about the Model
  • makemigrations_ts command - to generate "migrations" files with "TSDB" migration operations
  • maketsmeta command - that goes through all migrations and outputs all TSMeta class based on "migration operations" added manually - tooling to migrate from M.2 -> M.3

[daadu]

@schlunsen I have been busy with other projects recently.

Now that I am back! I have prepared this. Please give your feedback on it.

This could act as a central issue for planning/discussing new APIs etc.

This document will be updated as we go along.

[daadu]

@schlunsen Please also tag other active contributors for their views/feedback on it.

[fabienheureux]

Hi @daadu, would you say this project is ready for use on production ? Or is it still in a too early stage ?
Thanks !

[voarsh2]

@daa
Anyone able to have a look #56 ?
I've made a small branch change to add programmatic retention and compression.
If no one likes it, then no loss - I would rather maintain TimescaleDB separately if this isn't actively maintained. :/