* Extract function for capture
* Add capability to capture internal metrics
- This is behind an environment variable, we'll turn this on by default
on clickhouse-based installation.
- A special Org/Team gets created for this lazily, which is not accessible for
anyone. This makes avoiding issues with stats/billing/properties and easier.
* Capture some useful metrics into posthog
* Set up an unique constraint for for_internal_metrics organization
This avoids some nasty race conditions
* internal_metrics fixes
* Extract component OverviewTab
* New tabbing system for system status page
* Embed ClickhouseTab
* Generate a dashboard for internal metrics on demand
* More metrics
* Add more internal metrics, add tests
* Fix migration files
* Adjust copy
* Update task definitions
* Fix a typing issue
* Rename a component
* Migration fixup
* Try reset patch after test ends
* Clear lru_cache between tests
* Make PLUGIN_SERVER_INGESTION True by default
* Disable plugin server ingestion in tests by default
This is so that ingestion tests in here, which rely on old ingestion, don't fail. Plugin server ingestion itself is covered well in the plugin server repo.
* Enable PLUGIN_SERVER_INGESTION_HANDOFF = get_bool_from_env("PLUGIN_SERVER_INGESTION_HANDOFF
* Don't set PLUGIN_SERVER_INGESTION_HANDOFF in worker
* Add comments
* Remove _HANDOFF from PLUGIN_SERVER_INGESTION
* add stats counter for plugin server handoff, so we can verify events out and events in
* add whitelisted posthog and kea organizations
* disable ingestion this round --> first let's just check the plugin server can talk to kafka & clickhouse before sending real events to it
* enable ingestion in docker-compose.ch.yml
* eliminate bad merge
* async action event matching when using postgres plugin server ingestion (#3182)
* fix org
* remove _HANDOFF from topic
* add plugin_ to plugin server ingestion topic
* update plugin server to 0.7.0
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Allow Plugins configuration on Cloud
* Generate UUID in log_event for idempotency
* Make event UUID common for both EE event processing pipelines
* Restore frontend from master for changed direction
* Fix _process_event_ee
* Whitelist PostHog Inc. and KeaJS for Plugins on Cloud
* Test Plugins on Cloud organization whitelisting
* Fix FOSS
* Revert UUID changes (another PR)
* Update posthog-plugin-server to 0.6.0-beta
* Add relevant settings to KEYS in bin/plugins-server
* Log all EE events to events_handoff Kafka topic for plugin server
* Clean up settings
* Fix FOSS
* Don't introduce KAFKA_EVENTS_HANDOFF
* Add cosmetic newline
* Add DEBUG WAL print()
* add worker to the ecs config and deploy
* for testing
* pull from this branch for testing
* chain config renders
* split out events pipe
* Set is_heroku true because of heroku kafka
* update /e/ service to run on port 8001
* add 8001 to the container definition as well
* simplify
* test migrating w/ ecs task using aws cli
* split services
* typo in task def
* remove networkConfiguration from task definition
* duplicate
* task-def-web specific
* update events service name
* Handle base64 encoded kafka certs
* if it's empty then try to set it for env vars
* fix b64 decode call
* cleanups
* enable base64 encoding of keys for kafka
* depend on kafka-helper for deps
* reformat
* sort imports
* type fixes
* it's late, I can't type. typos.
* use get_bool_from_env
* remove debug bits. Trigger on master/main
* prettier my yaml
* add notes about ref in GA
* up cpu and memory
