0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-21 21:49:51 +01:00
Commit Graph

6 Commits

Author SHA1 Message Date
Ben White
7fd0fbb207
feat(cdp): Use cyclotron part 2 (#24746) 2024-09-10 09:05:30 +02:00
Harry Waye
9bf86629df
feat(async-migrations): add auto complete trivial migrations option (#11601)
* feat(async-migrations): add auto complete trivial migrations option

This change is to ensure that the `run_async_migrations --check` command
option is a light operation such that we can safely use this e.g. in an
init container to gate the starting of a K8s Pod.

Previous to this change, the command was also handling the
auto-completion of migrations that it deamed to not be required, via the
`is_required` migration function. Aside from this heaviness issue, it's
good to avoid having side effects from a check command.

One part of the code that I wasn't sure about is the version checking,
so any comments here would be much appreciated.

Note that `./bin/migrate` is the command we call from both ECS migration
task and the Helm chart migration job.

* update comment re versions

* wip

* wip

* wip

* update hobby

* rename to noop migrations
2022-09-01 14:43:09 +00:00
Tiina Turban
72042ee844
fix(async-migrations): bin/migrate to check not run async migrations (#8872) 2022-03-09 13:55:23 +01:00
Karl-Aksel Puulmann
c8d6b2225f
feat(sharding): add command to sync tables onto new nodes (#8912)
* feat(sharding): add command to sync tables onto new nodes

clickhouse-operator only syncs some tables onto new nodes. This new
command ensures that when adding new shards, they are automatically
synced up on redeploying

Note that there might be timing concerns here as resharding on altinity
cloud does not redeploy automatically. In practice however what this
means is that new nodes just won't ingest any data until another deploy

* Add test to the new command

* Improve non-replicated test
2022-03-08 12:50:49 +02:00
Yakko Majuri
3f85aa4c47
special migration -> async migration (#7686)
* special migration -> async migration

* rename frontend dir

* format

* rename SPECIAL_MIGRATION
2021-12-14 11:48:51 -03:00
Yakko Majuri
b56b141126
Special Migrations Runner (#7054 pt. 4) (#7446)
* add special migration definition and example

* types

* special migrations runner

* fix tests

* fix tests 2

* add clickhouse runner

* add temp fix for tests

* wip

* add special migrations api (#7448)

* wip new structure

* update example sourcing

* Update .gitignore

* yet another wip structure

* code quality

* cypress

* test docker image build

* implement resumable ops

* code quality

* add comments

* add warning

* add conditional requirements for migration

* add comment on is_required

* add dependency map

* wip dependencies and run migration on startup

* code quality

* fix bugs

* fix more bugs

* format

* types

* remove api from this branch

* types

* types

* update clickhouse script

* add is_migration_in_range util

* fix type

* fix runner

* add AUTO_START_SPECIAL_MIGRATIONS env var

* reset migration on start

* cleanup

* wip per op rollback

* prevent accidental status rollback

* add utils and definition test

* update example with rollback per op

* wip test special migration

* add first runner tests

* add runner tests

* add util for code paths

* fix test

* fix types

* fix types again

* cleanup

* cleanup

* add periodic healthcheck task tests

* remove unused imports

* safer row updates

* fix coalescing none checks

* code quality

* add docstrings

* fix

* fix deploys issue

* update scripts

* add delay

* address reviews

* address review comments

* address review comments

* address final comments

* fix import error

* fix tests

* remove unused imports

* fix tests

* fix task test

* remove unused return value

* remove unused special migrations code from migrate_clickhouse

* tweaks to support fresh deployments
2021-12-13 13:00:27 -03:00