mirror of
https://github.com/PostHog/posthog.git
synced 2024-11-24 18:07:17 +01:00
a27d452171
* feat(person-on-events): add option to delay all events This change implements the option outlined in https://github.com/PostHog/product-internal/pull/405 Here I do not try to do any large structural changes to the code, I'll leave that for later although it does mean the code has a few loose couplings between pipeline steps that probably should be strongly coupled. I've tried to comment these to try to make it clear about the couplings. I've also added a workflow to run the functional tests against both configurations, which we can remove once we're happy with the new implementation. Things of note: 1. We can't enable this for all users yet, not without the live events view and not without verifying that the buffer size is sufficiently large. We can however enable this for the test team and verify that it functions as expected. 2. I have not handled the case mentioned in the above PR regarding guarding against processing the delayed events before all events in the delay window have been processed. wip test(person-on-events): add currently failing test for person on events This test doesn't work with the previous behaviour of the person-on-events implementation, but should pass with the new delay all events behaviour. * add test for KafkaJSError behaviour * add comment re delay * add test for create_alias * chore: increase exports timeout It seems to fail in CI, but only for the delayed events enabled tests. I'm not sure why, but I'm guessing it's because the events are further delayed by the new implementation. * chore: fix test * add test for ordering of person properties * use ubuntu-latest-8-cores runner * add tests for plugin processEvent * chore: ensure plugin processEvent isn't run multiple times * expand on person properties ordering test * wip * wip * add additional test * change fullyProcessEvent to onlyUpdatePersonIdAssociations * update test * add test to ensure person properties do not propagate backwards in time * simplicfy person property tests * weaken guarantee in test * chore: make sure we don't update properties on the first parse We should only be updating person_id and asociated distinct_ids on first parse. * add tests for dropping events * increase export timeout * increase historical exports timeout * increase default waitForExpect interval to 1 second
18 lines
595 B
TypeScript
18 lines
595 B
TypeScript
export const waitForExpect = async <T>(fn: () => T | Promise<T>, timeout = 10_000, interval = 1_000): Promise<T> => {
|
|
// Allows for running expectations that are expected to pass eventually.
|
|
// This is useful for, e.g. waiting for events to have been ingested into
|
|
// the database.
|
|
|
|
const start = Date.now()
|
|
while (true) {
|
|
try {
|
|
return await fn()
|
|
} catch (error) {
|
|
if (Date.now() - start > timeout) {
|
|
throw error
|
|
}
|
|
await new Promise((resolve) => setTimeout(resolve, interval))
|
|
}
|
|
}
|
|
}
|