0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 18:26:15 +01:00
posthog/ee/clickhouse
Tim Glaser bf2c4429b5
Auto filter test accounts (#3492)
* WIP auto filter test accounts

* finish off

* Fix tests

* Non generic emails

* add list of generic emails

* Move location to below property filters

* Fix typescript errors

* as any

* fix tests

* filters

* fix tests

* Featureflag doesn't really makes sense for this feature

* fix tests

* fix test

* Add clickhouse + tests for each insight

* Fix lifecycle and paths

* Fix sessions

* Fix session tests

* fix sesisons test

* fix migrations

* fix migration chain

* refactor path & remove stale console.log

* adjust useAnchor & minor copy

* rename to avoid confusion with inline component

* test account filter tweaks

* fix filters

* hardcode

* Add tests for funnel trends

* Make generic emails super fast

* Fix migrations

* Default to false for now

* Default to false, remember a user's preference

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-03-11 18:16:38 +01:00
..
migrations Upload static cohort csv (#2932) 2021-01-15 11:19:31 +01:00
models Auto filter test accounts (#3492) 2021-03-11 18:16:38 +01:00
queries Auto filter test accounts (#3492) 2021-03-11 18:16:38 +01:00
sql Fix session tests (#3580) 2021-03-05 12:26:54 +01:00
test Run tests via pytest (#3417) 2021-02-24 09:32:44 +02:00
views Fix limit of events in CSV export (#3503) 2021-02-26 18:20:05 +01:00
__init__.py
client.py Run all connections through clickhouse connection pooler (#3495) 2021-02-26 12:54:35 -08:00
demo.py Populate a lone session recording for demo (#3128) 2021-01-29 14:28:55 +01:00
middleware.py Debug CH queries (#2666) 2020-12-07 16:06:14 +01:00
process_event.py Earlier IP anonymization (#3464) 2021-02-25 10:31:33 +01:00
README.md
util.py Upload static cohort csv (#2932) 2021-01-15 11:19:31 +01:00

Clickhouse Support (Enterprise Feature)

To accomodate high volume deployments, Posthog can use Clickhouse instead of Postgres. Clickhouse isn't used by default because Postgres is easier to deploy and maintain on smaller instances and on platforms such as Heroku.

Clickhouse Support works by swapping in separate queries and classes in the system for models that are most impacted by high volume usage (ex: events and actions).

Migrations and Models

The django_clickhouse orm is used to manage migrations and models. The ORM is used to mimic the django model and migration structure in the main folder.

Queries

Queries parallel the queries folder in the main folder however, clickhouse queries are written in SQL and do not utilize the ORM.

Tests

The tests are inherited from the main folder. The Clickhouse query classes are based off BaseQuery so their run function should work just as the Django ORM backed query classes. These classes are called with the paramterized tests declared in the main folder which allows the same suite of tests to be run with different implementations.

Views

Views contain Viewset classes that are not backed by models. Instead the views query Clickhouse tables using SQL. These views match the interface provide by the views in the main folder.