* feat: Have a local docker-compose stack for developing 100% in docker
* lol found a docker-compose bug where you can't have volumes created in root dir
* scale -> deploy.replicas
* don't forget to add image for asyncmigrationscheck
* env vars to each component
* Rename local to dev-full
* feat: remove version from docker compose to support new spec
* feat: simplify the docker-compose setup so we do less version coordinations
* update hobby bin
* bump docker-compose version for hobby for extends compat
* move ci to ubuntu-latest
* Revert "move ci to ubuntu-latest"
This reverts commit a0462adfec.
* use docker compose for github ci
* correct comments on base
* feat(async-migrations): add auto complete trivial migrations option
This change is to ensure that the `run_async_migrations --check` command
option is a light operation such that we can safely use this e.g. in an
init container to gate the starting of a K8s Pod.
Previous to this change, the command was also handling the
auto-completion of migrations that it deamed to not be required, via the
`is_required` migration function. Aside from this heaviness issue, it's
good to avoid having side effects from a check command.
One part of the code that I wasn't sure about is the version checking,
so any comments here would be much appreciated.
Note that `./bin/migrate` is the command we call from both ECS migration
task and the Helm chart migration job.
* update comment re versions
* wip
* wip
* wip
* update hobby
* rename to noop migrations
* Add persistent volumes to docker-compose-hobby
Per the discussion in https://github.com/PostHog/posthog/issues/10792, implemented the "Kessel Fix" in less than a parsec.
* Add warning text to user prompts to avoid data loss
Following discussion with PH team, we wanted to give users the information needed to properly manage the data in their installation and avoid potential data loss.
* refactor: Eliminate the `KAFKA_ENABLED` setting
* Remove dead code
* Consolidate plugin server test scripts and CI
* Fix CI command
* Remove Celery queues
* Rearrange test directories
* Update import paths
* chore(hobby deployments): various fixes
* default do not check versions for current hobby release
Co-authored-by: James Greenhill <fuziontech@gmail.com>
* chore: bump defaults for clickhouse to 22.3 across the board
* remove docker build instructions for clickhouse
* remove 21.11.11.1 from matrix
* prettier it
* feat(object_storage): add unused object storage with health checks
* only prompt debug users if object storage not available at preflight
* safe plugin server health check for unused object storage
* explicit object storage settings
* explicit object storage settings
* explicit object storage settings
* downgrade pip tools
* without spaces?
* like this?
* without updating pip?
* remove object_storage from dev volumes
* named volume on hobby
* lazily init object storage
* simplify conditional check
* reproduced error locally
* reproduced error locally
* object_storage_endpoint not host and port
* log more when checking kafka and clickhouse
* don't filter docker output
* add kafka to hosts before starting stack?
* silly cloud tests (not my brain)
* chore: Remove ClickHouse version 21.6 and add 22.3 for testing and docker-compose (21.6 is not in clickhouse dockerhub)
* update service requirements
* update arm64 compose as well
* Default to 21.11.11.1 and support multiple dockerhub repositories
* prettier .github/workflows/ci-backend.yml
Previously we were not defining volumes in ZooKeeper config for
docker-compose.hobby.yml.
We noticed the issue with the upgrade from 1.34.2 to 1.35.0 due to the
changes to ZooKeeper version. Previously I believe we were just stopping
and starting the ZooKeeper container. With no image changes this is
fine, we'll reuse the existing container. However, as we updated we end
up with a new container, without any of the previous data.
Coupled with the introduction of Replication which relies on ZooKeeper,
this brought down hobby deployments.
It looks like we also do not provide explicit volumes for e.g.
ClickHouse either, but I'm doing one fix at a time...
This change will not try to save anything that is in the existing
container. After running this command, data will need to be reinstated
with ClickHouse SYSTEM RESTORE REPLICA.
* feat(ClickHouse): upgrade Docker references to 21.11.11.1 (1/2)
* ci(backend): run tests on docker-compose changes
Co-authored-by: Harry Waye <harry@posthog.com>
* Fix docs for hobby and add upgrade instructions
* swallow errors from stop command (should error normally)
* persist env vars in docker-compose for hobby
* remove .hobby from docker-compose
* make file easier to test
* make env vars easier to template
* use docker-compose from git instead of curl
* test envsubst params
* default for sentry because exceptions
* tweaks to default logic
