significantly reduce logging
By default ALL logging is enabled, sometimes at trace level, which results in gigabytes of logs daily for an average website.
This commit reduces max log size and increases logging level to `error`.
* chore: just bump the max insert partitions for dev
* remove setting that is not yet available
* ruff
* Update query snapshots
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* add a few types
* context value adding
* first draft of a system that sort of works
* hacks to get it working
* fix alias
* simplify a lot
* set readonly to 2
* Update docker/clickhouse/config.xml
Co-authored-by: timgl <tim@glsr.nl>
* make it pass our list of urls
* better aliasing
* remove flimsy thing
* different type of input
* dead code
* refactor aapl_stock
* own it
* Update query snapshots
* more cleanup
---------
Co-authored-by: timgl <tim@glsr.nl>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* feat: Add Temporal to the dev and hobby stacks
* disable elastic for hobby because of resources
* checkpoint
* update requirements
* worker is up, but without the sandbox
* ensure temporal does not depend on elastic
* Feedbacked
* pip-compile dev
* mypy fixes
* add a bit of colorful logging
* add django temporal worker to the mix
* checkpoint for dev-full docker
* Working on docker-full, but checkpointing for now
* add migration bits for full
This speeds up tests that rely on clickhouse ingestion (plugin-server) significantly. The previous default was 7500, setting this shaves off seconds off of every single test
We don't set this in production due to the extra load it causes.
* chore(dev): use network mode host for docker-compose services
This removes the need to add kafka to /etc/hosts.
As far as I can tell this should be fine for peoples local dev except
they will be required to reset and re-migrate ClickHouse tables as they
will be trying to pull from `kafka` instead of `localhost`.
* remove ports from redis
* Update a few more references
* chore: bump defaults for clickhouse to 22.3 across the board
* remove docker build instructions for clickhouse
* remove 21.11.11.1 from matrix
* prettier it
* feat: wip add script to backfill persons and groups columns on events table
* updates
* cleanup + query id feature
* refactor
* fixes and add person tests
* add groups tests
* fix ci tests?
* test
* keep dictionaries around
* updates + massive sleep test?
* fixes
* updates
* fix for ch 21.6.5
* feat(ClickHouse): upgrade Docker references to 21.11.11.1 (1/2)
* ci(backend): run tests on docker-compose changes
Co-authored-by: Harry Waye <harry@posthog.com>
* m1 clickhouse support
* add clickhouse arm64 compilation dockerfile
* use clang-12 and 2 parallel builds
* working clickhouse docker image
* this needs to be committed out for CH to work... a known issue. WIP
* move things around
* move things around again
* add arm64 build scripts
* comment out broken line in DEBUG mode, still make it break production builds
* update readme
* timeless text
* add issue in comment
* way way way too much white space
* zookeeper restarts sometimes needed
* Make DDLs more friendly towards running on a cluster
* Use primary CLICKHOUSE host for migrations and DDL
* loose ends on person kafka create
* posthog -> cluster typo
* add cluster to KAFKA create for plugin logs
* Feed the type monster
* clusterfy local clickhouse
* test docker-compose backed github action
* run just clickhouse and postgres from docker-compose
* move option to between up and <services>
* posthog all the things
* suggest tests run on cluster
* posthog cluster for ci
* use deploy path for docker-compose
* fix for a clickhouse bug 🐛
* complete CH bug fixes
* 5439 the github actions pg configs
* remove CLICKHOUSE_DATABASE (handled automatically)
* update DATABASE_URL for code quality checks
* Missed a few DDLs on Person
* 5439 -> 5432 to please the people
* cleanup persons and use f strings <3 f strings
* remove auto parens
* Update requirements to use our fork of infi.clickhouse_orm
* fix person.py formatting
* Include boilerplate macros for a cluster
