* Pull in bitnami Kafka 2.8 files
* Make Kafka container multi-arch and upgrade to 2.8.2
* Use local Docker container for Kafka
* Expose Zookeeper locally
* Use ghcr prebuilt images
* feat: Add Temporal to the dev and hobby stacks
* disable elastic for hobby because of resources
* checkpoint
* update requirements
* worker is up, but without the sandbox
* ensure temporal does not depend on elastic
* Feedbacked
* pip-compile dev
* mypy fixes
* add a bit of colorful logging
* add django temporal worker to the mix
* checkpoint for dev-full docker
* Working on docker-full, but checkpointing for now
* add migration bits for full
* feat: remove version from docker compose to support new spec
* feat: simplify the docker-compose setup so we do less version coordinations
* update hobby bin
* bump docker-compose version for hobby for extends compat
* move ci to ubuntu-latest
* Revert "move ci to ubuntu-latest"
This reverts commit a0462adfec.
* use docker compose for github ci
* correct comments on base
This speeds up tests that rely on clickhouse ingestion (plugin-server) significantly. The previous default was 7500, setting this shaves off seconds off of every single test
We don't set this in production due to the extra load it causes.
* chore(dev): use network mode host for docker-compose services
This removes the need to add kafka to /etc/hosts.
As far as I can tell this should be fine for peoples local dev except
they will be required to reset and re-migrate ClickHouse tables as they
will be trying to pull from `kafka` instead of `localhost`.
* remove ports from redis
* Update a few more references
* refactor: Eliminate the `KAFKA_ENABLED` setting
* Remove dead code
* Consolidate plugin server test scripts and CI
* Fix CI command
* Remove Celery queues
* Rearrange test directories
* Update import paths
* chore: bump defaults for clickhouse to 22.3 across the board
* remove docker build instructions for clickhouse
* remove 21.11.11.1 from matrix
* prettier it
* feat(object_storage): add unused object storage with health checks
* only prompt debug users if object storage not available at preflight
* safe plugin server health check for unused object storage
* explicit object storage settings
* explicit object storage settings
* explicit object storage settings
* downgrade pip tools
* without spaces?
* like this?
* without updating pip?
* remove object_storage from dev volumes
* named volume on hobby
* lazily init object storage
* simplify conditional check
* reproduced error locally
* reproduced error locally
* object_storage_endpoint not host and port
* log more when checking kafka and clickhouse
* don't filter docker output
* add kafka to hosts before starting stack?
* silly cloud tests (not my brain)
* chore: Remove ClickHouse version 21.6 and add 22.3 for testing and docker-compose (21.6 is not in clickhouse dockerhub)
* update service requirements
* update arm64 compose as well
* Default to 21.11.11.1 and support multiple dockerhub repositories
* prettier .github/workflows/ci-backend.yml
* feat(ClickHouse): upgrade Docker references to 21.11.11.1 (1/2)
* ci(backend): run tests on docker-compose changes
Co-authored-by: Harry Waye <harry@posthog.com>
* Make clickhouse server image version configurable
* Attempt to use a reusable workflow for backend tests
* Try a composite action
* Add missing shell: bash
* Temporarily remove a step
* Cache id parameter
* Include the action when needing to rerun
* Move checking migrations to separate file
This didn't work in composite actions tests and slows down every
parallelized test execution for no reason. This should have been
separate in the first place.
* Rename job
* Improve check-migrations
* Start stack in the new job
* Remove shell
* Solve flaky test
* Remove unused file
* Create a test timings job
* Use checkout@v2 - https://github.com/cds-snc/github-actions/issues/23
* Ignore coverage files
* Save backend test durations
* Install pytest-split
* Clean up ci-backend
* Move actions code around
* wording improvement
* Save backend test durations
* Add a comment
* Make it possible to run tests with a different clickhouse version
* Remove debugging code
* Fix typo
* Boolean inputs please work
Fixes overeager test running as per https://github.com/actions/runner/issues/1483
Co-authored-by: PostHog Bot <hey@posthog.com>
