0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 17:54:41 +01:00
Commit Graph

8 Commits

Author SHA1 Message Date
Paolo D'Amico
066721e3c1
Stability & dev experience improvements (#2152) 2020-11-02 14:55:20 +00:00
James Greenhill
b64673ca4e
wire up the length to the proto message (#2089)
* wire up the length to the proto message

* we are so deep into the proto weeds we are using proto private methods
2020-10-28 17:41:13 -07:00
James Greenhill
601696456f
Start with a new topic (#2088) 2020-10-28 17:12:58 -07:00
James Greenhill
01099a5ffd
Provide required proto message length for our clickhouse overlords (#2087) 2020-10-28 16:48:05 -07:00
James Greenhill
83b5273113
Protobufize events to protect from malformed JSON (#2085)
* Protobuf all the things

* oops

* Protobufize events to protect from malformed JSON

* format the generated files (will need to remember this for future)

* format

* clean up kafka produce serializer

* fixes
2020-10-28 15:18:52 -07:00
Karl-Aksel Puulmann
e3bf0cb31d
Session recording on clickhouse, separate tables and retention cronjob (#2051)
* Add scheduled task to wipe session recordings

* Create a new table for session recording

* Save snapshot events to different table

* Use SessionRecordingEvent over Events everywhere

We can remove a ton of cruft this way as well

* Add missing signature

* Extract util from models/event

* Attempt to update ingest side of clickhouse session recording events

Note that it's using main kafka topic - not sure if a good idea.

* Get separate table in ch working for session recording events

* WIP: query sessions

* Make both session recording queries work

* Make linter happy

* Rebase migration

* Make tests work

* Apply a TTL to session recordings and other configuration:

- toYYYYMMDD partitioning should be smoother with TTL setup
- TTL achieves not needing to archive the data ourselves
- index_granularity will enable smaller reads per session_id
- ORDER BY clause is to make single session as well as time range query
  reasonable

* Convert retention cronjob to new model

* Add tests to process_event changes

* Add test for ee_capture change

* Fixup migration

* Make clickhouse tests drop/create session recording tables

* Make TTL not be there in tests

Otherwise writes get eaten by it during tests when mocking time

* Fix retention task

Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-10-28 21:22:16 +01:00
James Greenhill
7ab30a836c
Remove Omni-Person logic for ee (#1972)
* Remove Omni-Person logic for ee

* remove more omni person references
2020-10-21 14:06:45 -07:00
James Greenhill
b74d06a96a
Create a write ahead log for cloud event processing (#1962)
* Create a write ahead log for cloud event processing

* mypy fix

* if we are on app (ee) don't log to postgres

* don't disable writing to postgres
2020-10-21 20:35:07 +02:00