0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-25 11:17:50 +01:00
Commit Graph

472 Commits

Author SHA1 Message Date
Marius Andra
59eca63f30
fix: Revert "feat(hogvm): hogvm bytecode action matching in the plugin server" (#17036)
Revert "feat(hogvm): hogvm bytecode action matching in the plugin server (#16937)"

This reverts commit e0d2582e32.
2023-08-15 16:22:19 +01:00
Marius Andra
e0d2582e32
feat(hogvm): hogvm bytecode action matching in the plugin server (#16937) 2023-08-15 15:37:19 +02:00
Xavier Vello
abd6f82f64
feat(plugin-server): add slowlogs around ExportEventsBuffer flushes (#16966) 2023-08-10 14:44:16 +02:00
Marius Andra
8316edc87d
feat(hogvm): save bytecode for actions (#16932) 2023-08-10 11:13:03 +02:00
Xavier Vello
f5022f5a7e
chore(plugin-server): capture event uuid in timestamp parsing warnings (#16697) 2023-08-08 16:19:52 +02:00
Ben White
aff06a7683
fix: Improved metrics around blob ingester (#16774) 2023-07-26 16:42:45 +00:00
Ben White
d198daadf4
feat: Moved to gzipping on write stream instead of read (#16754) 2023-07-26 11:44:41 +02:00
Ben White
86acb8dd4a
fix: Improvements to S3 flushing. (#16750) 2023-07-25 14:03:59 +02:00
Ben White
6cbc5852a4
feat: Commit offsets more often (#16686) 2023-07-21 15:13:35 +02:00
Ben White
b024f91ac1
fix: Stop the drain check to see if that is the issue (#16718) 2023-07-21 12:03:56 +00:00
Ben White
99a99184d2
feat: Handle writestream backpressure (#16691) 2023-07-21 10:03:12 +02:00
Ben White
d8df34f4ab
feat: Replay events consumer (#16642) 2023-07-20 14:41:25 +00:00
Ben White
093476ea89
fix: Session high watermarks logic (#16656) 2023-07-19 10:37:15 +00:00
Ben White
7ef03c31b6
feat: Blob flush timeout handling (#16629) 2023-07-18 13:42:34 +02:00
Ben White
8b32bc05e9
feat: Blob ingester flush jitter (#16619) 2023-07-18 07:51:22 +00:00
Ben White
eac2bd3659
feat: Size based session flushing (#16610) 2023-07-18 09:17:25 +02:00
Ben White
6a13394ed9
chore: Added more topic - partition checking (#16606) 2023-07-17 12:43:17 +00:00
Ben White
2bef064c1e
fix: Commit logic for blob ingester (#16579) 2023-07-14 10:46:34 +02:00
Ben White
24088c8a73
fix: Blob ingester file deletion logic (#16561) 2023-07-13 13:39:44 +00:00
Marius Andra
d22bc0c4c7
fix(webhooks): split by dot (#16544) 2023-07-13 12:28:11 +00:00
Harry Waye
9ba9512f6e
test(recordings-blob): fix rebalance test (#16554)
I think maybe `ingester.sessions` changed somewhere along the time,
hopefully this fixes it.
2023-07-13 11:06:17 +00:00
Tiina Turban
f5304fc222
chore: nuke unused onSnapshot (#16531) 2023-07-13 12:03:59 +02:00
Harry Waye
9dcb8aa030
chore: fix plugin server tests for acitonMatcher (#16548)
* chore: fix plugin server tests for acitonMatcher

I broke these but there was an issue with CI so they were merges broken
but now maybe they are fixed?

* more fixes

* wip

* wip

* wip
2023-07-13 10:56:43 +01:00
Ben White
f3fedaa91d
fix: Session manager cleanup and deletion (#16521) 2023-07-13 11:48:49 +02:00
Paul D'Ambra
78a4ade041
chore: switch back to sync commit and gauge actual offset committed (#16540) 2023-07-12 21:30:09 +01:00
Harry Waye
06bd75ee1e
chore(plugin-server): we split onevent and webhooks consumers (#16511)
* Revert "Revert "chore(plugin-server): we split onevent and webhooks consumers" (#16510)"

This reverts commit 59af5b904d.

* remove capabiliity

* fix load actions

* fix typing

* fix tests

* wip
2023-07-12 15:44:21 +00:00
Ben White
cd2f7f398a
feat: Optimised blob storage team loading (#16486) 2023-07-12 10:21:06 +02:00
Ben White
0a84d018fc
feat: Reduce offset tracking (#16482) 2023-07-11 16:56:45 +02:00
Harry Waye
634472c3ad
fix: plugin mode string check (#16490)
* fix: plugin mode string check

Previously we had to keep these in sync, but now we can just use the
PluginServerMode enum directly.

* add test
2023-07-11 14:09:28 +00:00
Harry Waye
9f0cf9f40f
chore: make attachement test less flakey
We do this by making sure the plugin config and the attachement are
committed together.
2023-07-11 14:08:29 +00:00
Ben White
5a636f6bd1
feat: Optimise resource usage for blob ingester (#16478) 2023-07-11 15:11:36 +02:00
Paul D'Ambra
d068ba9410
feat: don't track byte size, it isn't useful enough (#16481)
* feat: track file line count not byte size

* just remove it
2023-07-11 13:23:01 +01:00
Harry Waye
8adac54130
chore(webhooks): remove abstractions from webhook consumer logic (#16418)
* chore(webhooks): remove abstractions from webhook consumer logic

Previously we were jumping through a few hoops to make webhook calls
e.g. still using the piscina abstraction, still using the runner code.
This commit removes those abstractions while still maintaining the
existing functionality wrt error handling and metrics gathering.

I'll leave further refactoring of the webhook consumer code to a
separate PR. For example, moving the statsd metrics to be based on
OpenMetrics instead. And further adding some tracing around key parts of
the webhook matching and firing logic.

* fix typing

* fix typing

* fix typing

* fix unit tests

* fix tests

* chore(plugin-server): simplify action manager deps

Previously we were passing in the kitchen sync, but the only
dependency is postgres. This should make it easier to e.g. refactor to
not need to load the kitchen sync on some deployments.

* chore(plugin-server): simplify hook commander deps

Previously we passed in DB which is a lot of stuff. Now we just pass in
the postgres pool.

* fix import
2023-07-10 11:02:04 +00:00
Harry Waye
4d454adbb9
chore(plugin-server): simplify action manager deps (#16426)
* chore(plugin-server): simplify action manager deps

Previously we were passing in the kitchen sync, but the only
dependency is postgres. This should make it easier to e.g. refactor to
not need to load the kitchen sync on some deployments.

* fix import
2023-07-07 16:12:53 +01:00
Harry Waye
12d0a29957
chore(plugin-server): simplify action matcher deps (#16429)
* chore(plugin-server): simplify action matcher deps

Specifically this only depends on postgres, so passing in the `DB`
object is unnecessary. This should make refactors easier to e.g. only
load the required dependencies when they are needed.

* pass only postgres to ActionMatcher
2023-07-07 15:02:46 +01:00
Harry Waye
16b31473d2
chore(plugin-server): only require postgres for org manager (#16425)
Previously we were passing the kitchen sink into this manager. This
makes it tricky to allow the manager to be used in other contexts.
2023-07-07 14:25:52 +01:00
Neil Kakkar
e8680f5646
fix(flags): Several minor fixes for experiments and flags (#16402) 2023-07-06 14:52:53 +00:00
Tiina Turban
b2b7eb4d26
feat: Add capabilities for webhooks - onEvent split (#16373) 2023-07-05 15:25:44 +02:00
Paul D'Ambra
efb77278b4
feat: async commits in the blob ingester (#16387)
* feat: async commits in the blob ingester

* Fix
2023-07-05 14:03:29 +01:00
Tiina Turban
a45c51c8c9
feat: Break up webhooks from onEvent (#16316) 2023-07-05 12:29:28 +02:00
Tiina Turban
876ce6a43c
fix: Nuke siteUrlManager reducing PG load (#16290) 2023-07-05 12:19:32 +02:00
Paul D'Ambra
0130bea5be
chore: add some realtime snapshot playback observability (#16363)
* chore: add some realtime snapshot playback observability

* fixes
2023-07-04 12:32:16 +00:00
Tiina Turban
17df8306bd
feat: Drop events by token and distinctID in plugin-server (#16308) 2023-07-03 13:10:21 +02:00
Tomás Farías Santana
72dd012cf2
fix: Update tests to use new number of retries (#16346) 2023-07-03 11:54:59 +02:00
Harry Waye
055776d461
chore(exports): try calling heartbeat a bit more often (#16295)
* chore(exports): try calling heartbeat a bit more often

Looks like we end up rebalancing often. Possibly because we're not
sending the heartbeats in time and the session timing out.

* wip

* fix tests
2023-06-29 13:55:31 +01:00
Paul D'Ambra
8a29cc679f
feat: estimate size of session on ingestion (#16241)
* migration to add the new column

* and populate it

* fix

* Update query snapshots

* test assertions

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-06-27 08:12:42 +01:00
Paul D'Ambra
4d0b51f764
fix: fake timer blob consumer tests (#16245)
* fix: fake timer blob consumer tests

* Update plugin-server/tests/main/ingestion-queues/session-recording/session-recordings-blob-consumer.test.ts

* remove unnecessary advance time
2023-06-26 12:52:01 +01:00
Paul D'Ambra
2623a77c05
chore: skip test to unblock CI (#16243)
YOLO to unblock CI
2023-06-26 12:04:13 +01:00
Ben White
34f6dde752
feat: New offset committing logic (#16220) 2023-06-23 11:18:35 +00:00
Ben White
ffdda1d392
feat: Realtime playback for new ingestion flow (#15627) 2023-06-23 12:39:07 +02:00