0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 18:26:15 +01:00
posthog/ee/clickhouse
Karl-Aksel Puulmann c5f19d8443
feat(time-to-see-data): ClickHouse query tagging (#12884)
* Decouple tagging from statsd metrics

* Add tags as JSON to log_comments in system.query_log

* tag by route_id

* Add insight_sync_execute utility

* Add filter.query_tags() and tag by number of dates

* Tag breakdown and trends with props

* rename, funnel correlations

* Tag actors queries

* Tag property_values query

* Tag get_earliest_timestamp query

* Tag paths query

* Tag retention query

* Tag stickiness query

* Tag trends formula query

* Update threading query a bit

* Tag retention actors query

* Tag lifecycle people query

* Tag funnel queries

* Add breakdown_by query tag

* Tag queries with more things

* Resolve a conflict

* Resolve circular imports

* Reformat

* Fix trends

* Remove corner casing for tags

* Tag graphs with user

* client_query_id tags

* More generic type

* Team access iteration

* Update test

* Add simple MV

* Fix date ranges

* Add comment to decorator
2022-11-22 13:38:57 +00:00
..
bin
materialized_columns
models
queries feat(time-to-see-data): ClickHouse query tagging (#12884) 2022-11-22 13:38:57 +00:00
test chore(time-to-see-data): Make ClickHouse query tagging thread-safe (#12861) 2022-11-21 09:16:22 +02:00
views
__init__.py
README.md

Clickhouse Support (Enterprise Feature)

To accomodate high volume deployments, Posthog can use Clickhouse instead of Postgres. Clickhouse isn't used by default because Postgres is easier to deploy and maintain on smaller instances and on platforms such as Heroku.

Clickhouse Support works by swapping in separate queries and classes in the system for models that are most impacted by high volume usage (ex: events and actions).

Migrations and Models

The django_clickhouse orm is used to manage migrations and models. The ORM is used to mimic the django model and migration structure in the main folder.

Queries

Queries parallel the queries folder in the main folder however, clickhouse queries are written in SQL and do not utilize the ORM.

Tests

The tests are inherited from the main folder. The Clickhouse query classes are based off BaseQuery so their run function should work just as the Django ORM backed query classes. These classes are called with the paramterized tests declared in the main folder which allows the same suite of tests to be run with different implementations.

Views

Views contain Viewset classes that are not backed by models. Instead the views query Clickhouse tables using SQL. These views match the interface provide by the views in the main folder.