0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 18:26:15 +01:00
posthog/ee/kafka_client/topics.py
Karl-Aksel Puulmann 78a787bc3b
Create and populate person_distinct_id2 table, add versioning to person_distinct_id (#7576)
* Migration to add version to person_distinct_id

* Update plugin-server type

* Use queueMessages instead of for loops

* Update distinct id versions in postgres

* Add commented out new query

* Add person_distinct_id2 table setup/migration

This will be used for more efficient person_distinct_id queries

* Avoid sharding person_distinct_id2 on cloud

* Write to new distinct ids topic

* Attempt to use version in tests

* Tests attempt 2

* Fixup version - dont send with all messages

* Flush kafka more frequently

* Actually fix tests

* Add another await

* Add partition to person_distinct_id2 table
2021-12-08 16:47:57 +02:00

16 lines
713 B
Python

# Keep this in sync with plugin-server/src/config/kafka-topics.ts
from posthog.settings import TEST
suffix = "_test" if TEST else ""
KAFKA_EVENTS_PLUGIN_INGESTION: str = f"events_plugin_ingestion{suffix}" # can be overridden in settings.py
KAFKA_EVENTS = f"clickhouse_events_proto{suffix}"
KAFKA_PERSON = f"clickhouse_person{suffix}"
KAFKA_PERSON_UNIQUE_ID = f"clickhouse_person_unique_id{suffix}"
KAFKA_PERSON_DISTINCT_ID = f"clickhouse_person_distinct_id{suffix}"
KAFKA_SESSION_RECORDING_EVENTS = f"clickhouse_session_recording_events{suffix}"
KAFKA_PLUGIN_LOG_ENTRIES = f"plugin_log_entries{suffix}"
KAFKA_DEAD_LETTER_QUEUE = f"events_dead_letter_queue{suffix}"
KAFKA_GROUPS = f"clickhouse_groups{suffix}"