0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-22 08:40:03 +01:00
Commit Graph

8 Commits

Author SHA1 Message Date
Brett Hoerner
30bafdd382
chore(plugin-server): kafka ack cleanup and metric (#21111)
* cleanup: remove unused team arg from registerLastStep

* cleanup: rename promises to ackPromises to make it more clear thats what they are

* cleanup(plugin-server): make waitForAck explicit/required

* add Kafka produce/ack metrics

* Clarify Kafka produce metric/labels
2024-03-25 13:01:15 +00:00
Brett Hoerner
d68fa14f10
chore(plugin-server): run node-rdkafka with cooperative rebalancing patched in (#17747)
* Remove node-rdkafka-acosom

* Add node-rdkafka

* Replace node-rdkafka-acosom imports with node-rdkafka

* Patch node-rdkafka with changes from https://github.com/PostHog/node-rdkafka/

* Add patch directions to README
2023-10-10 08:13:05 -06:00
Harry Waye
2f9e2928fe
chore(plugin-server): use librdkafka producer everywhere (#15314)
* chore(plugin-server): use librdkafka producer everywhere

We say some 10x improvements in the throughput for session recordings.
Hopefully there will be more improvements here as well, although it's a
little less clear cut.

I don't try to provide any improvements in guarantees around message
production here.

* we still need to enable snappy for kafkajs
2023-05-04 13:02:44 +00:00
Harry Waye
96fe16fd3c
chore(recordings): use cooperative-sticky rebalance strategy (#15260)
Revert "revert(recordings): use cooperative-sticky rebalance strategy (#15211)"

This reverts commit a40f01138e.
2023-04-26 13:09:13 +00:00
Harry Waye
a40f01138e
revert(recordings): use cooperative-sticky rebalance strategy (#15211)
Revert "chore(recordings): use cooperative-sticky rebalance strategy (#15197)"

This reverts commit 3eddb96b9b.
2023-04-24 15:06:33 +00:00
Harry Waye
3eddb96b9b
chore(recordings): use cooperative-sticky rebalance strategy (#15197)
* chore(recordings): use cooperative-sticky rebalance strategy

This should make rebalances and lag during deploys a little less
painful. I'm setting this as the globally used strategy, when we e.g.
want to use another strategy for a specific consumer group, we can make
this configurable.

* disable rebalance_callback

* use node-rdkafka-acosom fork instead, for cooperative support
2023-04-24 13:25:24 +00:00
Harry Waye
01da91d1e2
test: remove explicit dependencies from functional tests (#14471)
* test: remove explicit dependencies from functional tests

* Install pnpm in devcontainer
2023-03-01 12:03:54 +00:00
Harry Waye
a4a3a0c902
test(plugin-server): use librdkafka for functional tests (#14468)
* test(plugin-server): use librdkafka for functional tests

While trying to port the session recordings to use node-librdkafka I
found it useful to first implement it in the functional tests.

* use obj destructuring to make calls more self explanatory
2023-03-01 11:03:13 +00:00