Karl-Aksel Puulmann
30f59dfef2
feat(historical-exports): Include historical exports finishing in activity log ( #12068 )
...
* Add a is_system flag to activity logs
* Allow writing activity logs from within plugin-server
* Make changes object non-required
* Render system user information
* Log when export finishes or fails in plugin activity log
* Update activityLogLogic.plugin.test.tsx
2022-10-07 13:25:16 +03:00
Karl-Aksel Puulmann
02bae96be5
fix(historic-exports): $autocapture classnames parsing ( #11626 )
...
Re-ingesting historically exported autocaptured events currently fails: https://sentry.io/organizations/posthog2/issues/3552913543/?query=is%3Aunresolved+level%3Aerror
This is because of attr__class being sometimes there under attributes, other times not, so $elements.attr_class could be either a string or an array when leaving posthog in an export.
Rather than fix this at the export, making the parsing logic in ingestion more flexible
2022-09-02 16:28:15 +03:00
Yakko Majuri
cb689c9129
feat: add retries for enqueuing graphile jobs ( #11561 )
...
* feat: add retries for enqueuing graphile jobs
* fix tests
* fix tests
* address review
* fix build
* add back mock clear?
* bring back old approach
2022-09-01 14:54:32 -03:00
Tiina Turban
c6b1da5932
fix: hide initial referrer as event property ( #11536 )
2022-08-30 18:07:02 +02:00
Michael Matloka
7bd3cac2f5
refactor(plugin-server): Unify event types ( #10612 )
...
* Simplify Event, ClickHouseEvent, PreIngestionEvent, IngestionEvent
* Unify `ClickhouseEventKafka` with `RawEvent`
* Fix imports
* Eliminate PostgresSessionRecordingEvent
* Parse `Event.elements_chain` too
* Update process-event.test.ts
* Update tests
* Make `IngestionEvent['timestamp']` consistent
* Update tests
* Restore `PreIngestionEvent` vs. `PostIngestionEvent` split
* Update worker.test.ts
* Improve typing a bit
* Update tests to work with mandatory `DateTime`
* Remove ClonableIngestionEvent
* Rename RawEvent -> RawClickHouseEvent
* Rename Event -> ClickHouseEvent
* Update prepareEventStep tests
* Update convertToIngestionEvent behavior back to master
* Update tests to compile
* Use branded types for ISO/Clickhouse timestamp string disambiguation
* Test for parseRawClickHouseEvent()
* Update each-batch tests
* Tests for clickHouseTimestampToDateTime()
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
2022-08-15 10:54:09 +03:00
Karl-Aksel Puulmann
6457f0296b
fix(ingestion): Change overrides order when parsing Kafka messages ( #10998 )
...
* Move formPluginEvent
* Work around risky behavior
Previously, users could override some important event fields by passing
values in their payload. This bug was introduced way back in https://github.com/PostHog/plugin-server/pull/34
This bug indirectly caused the following sentry errors:
- https://sentry.io/organizations/posthog2/issues/3289550563/?project=6423401&query=is%3Aunresolved+level%3Aerror
- https://sentry.io/organizations/posthog2/issues/3455742732/?project=6423401&query=is%3Aunresolved+level%3Aerror
- https://sentry.io/organizations/posthog2/issues/3382895905/?project=6423401&query=is%3Aunresolved+level%3Aerror
One area I'm unsure on is specifically `ip` field and its expected
behavior, but looking at old code from 2020 it seems we always took the
ip from request rather than looking at event body.
2022-07-27 13:26:50 +03:00
Michael Matloka
313226838c
revert: revert: Revert person properties updates refactor ( #10349 )
...
* Revert "revert: Revert person properties updates refactor (#10348 )"
This reverts commit 6b3c4691b3
2022-06-20 09:49:11 +03:00
Neil Kakkar
6b3c4691b3
revert: Revert person properties updates refactor ( #10348 )
2022-06-17 17:48:20 +02:00
Karl-Aksel Puulmann
d6ec3aedc6
refactor(plugin-server): person state updating ( #10321 )
...
* Remove some excessive call signatures
* refactor: move property sanitization outside of .capture
* Move event sanitization into event sanitization logic
* Move person creation/updating logic outside of capture/createSnapshot
* refactor: remove personManager from arguments
* refactor: remove various properties from arguments
* Update `handleIdentifyOrAlias` signature
* refactor: inline timeoutGuard into personStateManager
* refactor: prefix pipeline steps with indexes
* Extract timestamp parsing logic from process-event.ts
* refactor: move timestamp tests over from process-event.ts
* refactor: update process-event.test.ts
* refactor: person-state-manager -> person-state
* Move sanitizeEvent to a more suitable module
* Fix some process-event tests
2022-06-17 09:17:08 +03:00
Karl-Aksel Puulmann
6a6faf600e
fix(plugin-server): chain-to-elements regular expression behavior ( #10170 )
...
* refactor tests to be more extensible
* Move elements-related code to separate file
* Copy over tests for element chain
* Handle `undefined` match group
As this test from python indicates, the right-most match group may be
empty. In javascript the behavior for this is different from python and
the match group may be `undefined`
* Update import
* Fix http capabilities
2022-06-07 15:13:25 +03:00
