* fix: dont OOM deleting large teams/organizations
Another issue plaguing team deletion was that django ORM loads all
objects for a team into memory. If a team had >200k people, that would
cause the process to fail.
We now instead delete via a single sql statement.
* Change crontab parsing logic
* Cron for deleting clickhouse data
* Delete data conditionally
* Add schedule to tasks
* refactor: Eliminate the `KAFKA_ENABLED` setting
* Remove dead code
* Consolidate plugin server test scripts and CI
* Fix CI command
* Remove Celery queues
* Rearrange test directories
* Update import paths
* Make settings dynamic for MATERIALIZED_COLUMNS_ENABLED
* migrate MATERIALIZED_COLUMNS_ENABLED to this and test
* allow tests that use materialized columns to hit django model for settings
* nicer installed apps config
* update snapshot from test
* make test_property more deterministic
Co-authored-by: James Greenhill <fuzionech@gmail.com>
Co-authored-by: yakkomajuri <yakko.majuri@gmail.com>
* Add command for materializing columns
Expecting this to get used in both dev as well as when improving
upstream installations
* No clashes in tests
* Solve for feedback
* Comments for clarity
* Avoid naming collisions when materializing columns
1. Prefix person properties differently. Mixing them up can break
ambigious column issues
2. When name already exists, suffix with random junk :)
* Implement analyze.py
* Add `suggested_columns_to_materialize`
* Add code to backfill a materialized column
* Add tests for backfilling data
* Cleanup
* Add tests for analyze
* WIP: Crontab for materializing columns
* Nooped task for materializing properties
* Use mutations_sync=0 for column tests
* Add comment
* Add migration marking existing columns as materialized with a comment
* Add method to allow looking up materialized columns
* Add test for fetching and creating materialized columns
* Disable caching in tests
* Remove some callsites to CLICKHOUSE_DENORMALIZED_PROPERTIES
* Fixup caching logic
* Add (currently broken) test for denormalizing person properties
* Update funnel event query to work with new functionality
* Update other tests/queries
* Remove now-unused setting
* Enable PLUGIN_SERVER_INGESTION_HANDOFF = get_bool_from_env("PLUGIN_SERVER_INGESTION_HANDOFF
* Don't set PLUGIN_SERVER_INGESTION_HANDOFF in worker
* Add comments
* Remove _HANDOFF from PLUGIN_SERVER_INGESTION
* add stats counter for plugin server handoff, so we can verify events out and events in
* add whitelisted posthog and kea organizations
* disable ingestion this round --> first let's just check the plugin server can talk to kafka & clickhouse before sending real events to it
* enable ingestion in docker-compose.ch.yml
* eliminate bad merge
* async action event matching when using postgres plugin server ingestion (#3182)
* fix org
* remove _HANDOFF from topic
* add plugin_ to plugin server ingestion topic
* update plugin server to 0.7.0
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Fix unix-dgram build
* Add per-organization whitelisting of plugins API on Cloud
* Add migration
* Update can_install_plugins_via_api calls in serializer
* Remove unused type: ignore
* Update PluginSerializer for organization_id
* Make Plugin.organization nullable
* Fix can_install_plugins_via_api
* Update pluginsLogic.ts
* Adjust can_configure_plugins_via_api for organization
* Don't include organization_id in serializer
* Set default Plugin.organization to first Organization in instance
* Fix PluginConfigSerializer.create
* Test that other orgs' plugins can't be accessed (as if they didn't exist, 404)
* fix typo
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Polish system status page
* Show system health badge to is_staff users on Cloud
* Update Cypress test
* Correct inconsistent scene name
* Use RDBMS enum instead of constants
* Add "Analytics database in use" to System Status
* Fix typing
