* cleanup: remove unused team arg from registerLastStep
* cleanup: rename promises to ackPromises to make it more clear thats what they are
* cleanup(plugin-server): make waitForAck explicit/required
* add Kafka produce/ack metrics
* Clarify Kafka produce metric/labels
There is a race condition in these tests where the consumer isn't
consuming in time to pick up bad messages, so we ensure that we set the
offsets to the earliest messages.
* test(plugin-server): use librdkafka for functional tests
While trying to port the session recordings to use node-librdkafka I
found it useful to first implement it in the functional tests.
* use obj destructuring to make calls more self explanatory
* chore(plugin-server): Add metrics for time of last processed message
Previously we have been alerting on Kafka consumer group offset lag.
However, really we care about the delay between messages being written
to Kafka and being processed by the plugin server.
By adding the last processed timestamp, as a gauge, we can then alert on
if that time and now is greater than a threshold.
This alert would not require the plugin-server to be up to trigger, just
that there be some time registered so it handles complete failure also.
For the case that there are no messages past the committed offsets, we
will end up triggering the alert if we do not also take into
consideration the production rate into the topic.
* wip
* wip
* fix imports order
* fix group id
* Add and use waitForExpect instead
* remove yarn.lock
* move comment
* wip
* chore(plugin-server): split functional tests into feature based files
This is intended to make it more obvious what we are testing, and to try
and identify the major themes of the plugin-server functionality.
As a by product it should make things more parallelizable for jest as
the tests in different files will be isolated, runnable in separate
workers.
* use random api token, avoid db constraints
* make tests silent
* format
* chore: set number of jest workers
These tests should be pretty light given they just hit other APIs and
don't do much themselves. Memory could be an issue on constrained
environments. We shall see.
