significantly reduce logging
By default ALL logging is enabled, sometimes at trace level, which results in gigabytes of logs daily for an average website.
This commit reduces max log size and increases logging level to `error`.
* chore: just bump the max insert partitions for dev
* remove setting that is not yet available
* ruff
* Update query snapshots
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* add a few types
* context value adding
* first draft of a system that sort of works
* hacks to get it working
* fix alias
* simplify a lot
* set readonly to 2
* Update docker/clickhouse/config.xml
Co-authored-by: timgl <tim@glsr.nl>
* make it pass our list of urls
* better aliasing
* remove flimsy thing
* different type of input
* dead code
* refactor aapl_stock
* own it
* Update query snapshots
* more cleanup
---------
Co-authored-by: timgl <tim@glsr.nl>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
This speeds up tests that rely on clickhouse ingestion (plugin-server) significantly. The previous default was 7500, setting this shaves off seconds off of every single test
We don't set this in production due to the extra load it causes.
* chore(dev): use network mode host for docker-compose services
This removes the need to add kafka to /etc/hosts.
As far as I can tell this should be fine for peoples local dev except
they will be required to reset and re-migrate ClickHouse tables as they
will be trying to pull from `kafka` instead of `localhost`.
* remove ports from redis
* Update a few more references
* feat: wip add script to backfill persons and groups columns on events table
* updates
* cleanup + query id feature
* refactor
* fixes and add person tests
* add groups tests
* fix ci tests?
* test
* keep dictionaries around
* updates + massive sleep test?
* fixes
* updates
* fix for ch 21.6.5
* feat(ClickHouse): upgrade Docker references to 21.11.11.1 (1/2)
* ci(backend): run tests on docker-compose changes
Co-authored-by: Harry Waye <harry@posthog.com>
* Make DDLs more friendly towards running on a cluster
* Use primary CLICKHOUSE host for migrations and DDL
* loose ends on person kafka create
* posthog -> cluster typo
* add cluster to KAFKA create for plugin logs
* Feed the type monster
* clusterfy local clickhouse
* test docker-compose backed github action
* run just clickhouse and postgres from docker-compose
* move option to between up and <services>
* posthog all the things
* suggest tests run on cluster
* posthog cluster for ci
* use deploy path for docker-compose
* fix for a clickhouse bug 🐛
* complete CH bug fixes
* 5439 the github actions pg configs
* remove CLICKHOUSE_DATABASE (handled automatically)
* update DATABASE_URL for code quality checks
* Missed a few DDLs on Person
* 5439 -> 5432 to please the people
* cleanup persons and use f strings <3 f strings
* remove auto parens
* Update requirements to use our fork of infi.clickhouse_orm
* fix person.py formatting
* Include boilerplate macros for a cluster
