Previously we were fetching everything from the person table, but we
only actually care about the ids.
I've added a couple of other TODOs in here of obvious improvements. I'm
not sure what this will do for our write IO issues, but hopefully will
relieve some load and data transfer.
* perf(cohorts): reduce select batch size, and sleep 1 between
Previously we were hammering the database, see
https://github.com/PostHog/product-internal/issues/256 for details. Now
we:
1. reduce the batch by 5 times
2. have a little sleep(1) between batches
We also add a little bit of logging for instance in failure cases.
* include stack trace info in warning
* just reraise so we don't lose stack trace
* use monotonic for timings, not time
* update snapshots
* add test
* update func
* add include and settings
* diff method
* some prelim changes to make backend work
* add tests and logic for handling groups
* setup basic mixin parsing
* add tests
* types and switchover
* adjustment
* update snapshots
* update more snapshots
* update more snapshots
* update more snapshots
* typing
* snapshots
* clean up TODOs and types
* separate to_dict
* another property dict fix
* use operator dependening on condition
* fix tests
* update tests
* remove dummy condition
* update tests
* change name to groups
* another label
* rename filter_group to property_group
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* Remove flag `dashboard-redesign` and purge old code
* Update dashboard.js Cypress suite partially
* Reflect #8493 in moved DashboardHeader
* Make diff saner by not renaming `LemonDashboardHeader.tsx` yet
* Make editing dashboard grid react faster
* Remove redundant CSS
* Update dashboard.js Cypress suite completely
* Update insights.js
* Update EditableField.scss
* Use `Insight.last_modified_at` instead of `updated_at`
* Deprecate `Insight.updated_at`
* feat(healthchecks): add clickhouse as a dependency
* feat(healthchecks): add celery broker healthcheck
* feat(healthchecks): add cache healthcheck, plus some misc updates
* add missing typing
* add missing typing
* add role cast
* Add logging for kafka connectivity check
* actually add healthcheck as middleware
* fix postgres failure check
* simplify postgres migration return value
* verboise -> verbose
Co-authored-by: Guido Iaquinti <4038041+guidoiaquinti@users.noreply.github.com>
* put interface validation at top of view
* remove unused imports
* show resp.content on status failures
* test(healthchecks): set log level to debug to pytest displays it
* restore logger level
* Bring up redis for backend ci tests
* up redis in non-cloud tests
Co-authored-by: Guido Iaquinti <4038041+guidoiaquinti@users.noreply.github.com>
* test(snapshots): update clickhouse snapshot comments
It looks like Clickhouse has been removed from clickhouse sql comments,
but the snapshots haven't been updated. Not sure how this got past CI.
* missed a couple
* version insights
* version and lock update
* make sure all tests work
* restore exception
* fix test
* fix test
* add specific id
* update plugin server test utils
* cleanup
* match filtering
* use timestamp comparison
* make tests work
* one more test field
* fix more tests
* more cleanup
* lock frontend when updating and restore refresh
* pass undefined
* add timestamp to background update
* use incrementer
* add field
* snapshot
* types
* more cleanup
* update tests
* remove crumbs
* use expressions
* make nullable
* batch delete
* fill null for static cohorts
* batch_delete
* typing
* remove queryset function
* fetch person properties via clickhouse
* cleaner queries
* fix some types
* parse up to 100k persons with this prop, add back counts
* ignore something that's bound to error anyway
* no point making it too flat now
* revert flattening of lists
* add statsd
* ignore type
* add back conversion
* mypy
* better mypy
* refactor
* add support for Experiment groups
* update styling
* incl groups in property selection
* set participant type and respect FF group type on backend
* remove aggregation select
