* test(plugin-server): use librdkafka for functional tests
While trying to port the session recordings to use node-librdkafka I
found it useful to first implement it in the functional tests.
* use obj destructuring to make calls more self explanatory
* ci(plugin-server): wait for server to be listening before running tests
The export tests seem to be very flaky. Looking through the logs on
GitHub Actions it looks like there are some http connection refused
errors, so I'm additionally adding a wait for the listening event to be
emitted to try to remove some racy behaviour.
* add better timeout handling
* feat(person-on-events): add option to delay all events
This change implements the option outlined in
https://github.com/PostHog/product-internal/pull/405
Here I do not try to do any large structural changes to the code, I'll
leave that for later although it does mean the code has a few loose
couplings between pipeline steps that probably should be strongly
coupled. I've tried to comment these to try to make it clear about the
couplings.
I've also added a workflow to run the functional tests against both
configurations, which we can remove once we're happy with the new
implementation.
Things of note:
1. We can't enable this for all users yet, not without the live events
view and not without verifying that the buffer size is sufficiently
large. We can however enable this for the test team and verify that
it functions as expected.
2. I have not handled the case mentioned in the above PR regarding
guarding against processing the delayed events before all events in
the delay window have been processed.
wip
test(person-on-events): add currently failing test for person on events
This test doesn't work with the previous behaviour of the
person-on-events implementation, but should pass with the new delay all
events behaviour.
* add test for KafkaJSError behaviour
* add comment re delay
* add test for create_alias
* chore: increase exports timeout
It seems to fail in CI, but only for the delayed events enabled tests.
I'm not sure why, but I'm guessing it's because the events are further
delayed by the new implementation.
* chore: fix test
* add test for ordering of person properties
* use ubuntu-latest-8-cores runner
* add tests for plugin processEvent
* chore: ensure plugin processEvent isn't run multiple times
* expand on person properties ordering test
* wip
* wip
* add additional test
* change fullyProcessEvent to onlyUpdatePersonIdAssociations
* update test
* add test to ensure person properties do not propagate backwards in time
* simplicfy person property tests
* weaken guarantee in test
* chore: make sure we don't update properties on the first parse
We should only be updating person_id and asociated distinct_ids on first
parse.
* add tests for dropping events
* increase export timeout
* increase historical exports timeout
* increase default waitForExpect interval to 1 second
* chore(plugin-server): split functional tests into feature based files
This is intended to make it more obvious what we are testing, and to try
and identify the major themes of the plugin-server functionality.
As a by product it should make things more parallelizable for jest as
the tests in different files will be isolated, runnable in separate
workers.
* use random api token, avoid db constraints
* make tests silent
* format
* chore: set number of jest workers
These tests should be pretty light given they just hit other APIs and
don't do much themselves. Memory could be an issue on constrained
environments. We shall see.
