0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 09:16:49 +01:00
Commit Graph

2 Commits

Author SHA1 Message Date
Marius Andra
dd7e38c5b5
Clickhouse Elements Dedup (based on master) (#1698)
* Use ReplacingMergeTree for elements, remove element_groups and use elements_hash as a virtual "pk"

* remove unused ELEMENT_GROUP_TABLE_SQL

* merge fixes

* use redis cache to avoid writing duplicate elements to clickhouse

* move fakeredis to requirements.txt

* add team_id to cache key

* remove elements_group kafka table references

* add elements_hash to clickhouse element serializer

* fix cache key

* rename few keys

* add test runner to ease pycharm dev

* fix a some mypy errors

* remove typo

Co-authored-by: Eric <eeoneric@gmail.com>
2020-09-24 06:47:28 -04:00
James Greenhill
adeb7694cc
Publish events to Kafka for consumption (#1644)
* Publish events to Kafka for consumption

* Commit avro idl's for event schemas

* convert client to use github.com/dpkp/kafka-python

* events loaded into clickhouse from Kafka

* remove cruft

* Publish events to Kafka for consumption

* convert client to use github.com/dpkp/kafka-python

* remove cruft

* include kafka migrations

* bugfixes for migrations

* use constants for consistency

* wrap up local migrations

* small fixes

* tune ups
2020-09-15 20:04:38 -07:00