Tomás Farías Santana
5d7900dfd5
feat: Add sent-at property during normalizeEvent ( #15151 )
2023-04-21 11:31:59 +02:00
Yakko Majuri
90f1b16285
feat(ingestion): remove postgres dependency from capture endpoint ( #12802 )
...
* add support for token field in kafka message
* formPipelineEvent
* rename pipeline files according to new order
* wip team_id and anonymize ips
* conditional handlers and tests
* some plugin server fixes
* fix capture bug
* fix
* more fixes
* fix capture tests
* pipeline update
* fix + investigate database resets
* fix import order
* testing and typing updates
* add test for capture endpoint
* testing
* python typing
* plugin server test
* functional test
* fix test
* another fix
* make sure no team ids clash in tests
* fix
* add more metrics and logs
* cache nulls
* updates
* add more metrics
2022-11-23 09:55:26 -03:00
Yakko Majuri
4f372c05f9
feat(plugin-server): simplify groups caching ( #12403 )
...
* refactor(plugin-server): simplify groups caching
* add multi groups test
* remove comments
* fix type, add debug
* fix
* stringify
* add groups created_at to types
* more test fixes
* use the right clickhouse timestampo format
* update created at to ch format in tests
* finally
* more fixes
2022-10-25 15:35:47 -03:00
Michael Matloka
7bd3cac2f5
refactor(plugin-server): Unify event types ( #10612 )
...
* Simplify Event, ClickHouseEvent, PreIngestionEvent, IngestionEvent
* Unify `ClickhouseEventKafka` with `RawEvent`
* Fix imports
* Eliminate PostgresSessionRecordingEvent
* Parse `Event.elements_chain` too
* Update process-event.test.ts
* Update tests
* Make `IngestionEvent['timestamp']` consistent
* Update tests
* Restore `PreIngestionEvent` vs. `PostIngestionEvent` split
* Update worker.test.ts
* Improve typing a bit
* Update tests to work with mandatory `DateTime`
* Remove ClonableIngestionEvent
* Rename RawEvent -> RawClickHouseEvent
* Rename Event -> ClickHouseEvent
* Update prepareEventStep tests
* Update convertToIngestionEvent behavior back to master
* Update tests to compile
* Use branded types for ISO/Clickhouse timestamp string disambiguation
* Test for parseRawClickHouseEvent()
* Update each-batch tests
* Tests for clickHouseTimestampToDateTime()
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
2022-08-15 10:54:09 +03:00
Karl-Aksel Puulmann
6457f0296b
fix(ingestion): Change overrides order when parsing Kafka messages ( #10998 )
...
* Move formPluginEvent
* Work around risky behavior
Previously, users could override some important event fields by passing
values in their payload. This bug was introduced way back in https://github.com/PostHog/plugin-server/pull/34
This bug indirectly caused the following sentry errors:
- https://sentry.io/organizations/posthog2/issues/3289550563/?project=6423401&query=is%3Aunresolved+level%3Aerror
- https://sentry.io/organizations/posthog2/issues/3455742732/?project=6423401&query=is%3Aunresolved+level%3Aerror
- https://sentry.io/organizations/posthog2/issues/3382895905/?project=6423401&query=is%3Aunresolved+level%3Aerror
One area I'm unsure on is specifically `ip` field and its expected
behavior, but looking at old code from 2020 it seems we always took the
ip from request rather than looking at event body.
2022-07-27 13:26:50 +03:00
Michael Matloka
313226838c
revert: revert: Revert person properties updates refactor ( #10349 )
...
* Revert "revert: Revert person properties updates refactor (#10348 )"
This reverts commit 6b3c4691b3
2022-06-20 09:49:11 +03:00
Neil Kakkar
6b3c4691b3
revert: Revert person properties updates refactor ( #10348 )
2022-06-17 17:48:20 +02:00
Karl-Aksel Puulmann
d6ec3aedc6
refactor(plugin-server): person state updating ( #10321 )
...
* Remove some excessive call signatures
* refactor: move property sanitization outside of .capture
* Move event sanitization into event sanitization logic
* Move person creation/updating logic outside of capture/createSnapshot
* refactor: remove personManager from arguments
* refactor: remove various properties from arguments
* Update `handleIdentifyOrAlias` signature
* refactor: inline timeoutGuard into personStateManager
* refactor: prefix pipeline steps with indexes
* Extract timestamp parsing logic from process-event.ts
* refactor: move timestamp tests over from process-event.ts
* refactor: update process-event.test.ts
* refactor: person-state-manager -> person-state
* Move sanitizeEvent to a more suitable module
* Fix some process-event tests
2022-06-17 09:17:08 +03:00
