* add window-id to ingestion
* move window_id into snapshot data
* Add tests
* add window_id to snapshot_data only if included in the event
* add window_id to session_recording_events table
* revert ph-js downgrade
* fix data generator
* remove partition change
* update window_id defaults for tests
* missed one window_id default
* rename migration for conflict
* Migration to use materialized columns for groups
Workaround for https://github.com/PostHog/posthog/issues/6422
* Use groups materialized columns in queries
* Update mat column creation tests
* Simplify aggregation_target_field
* Fix migration
* Update snapshots
* smallest change to make aggregation work
* address comments
* add snapshot
* move function to groups model
* update funnel snapshot
* rename person_id to aggregation_target
* update snapshots as well
* dont support persons query mods for now
* update snapshot
* make array orders deterministic
* Improve process_math
* Add test for overlapping group keys
* Improve event query tests
* Add test for filtering by person properties together with groups
* Avoid flaky tests due to cohort_id changing
* Update queries and snapshots
* Add groups stuff
* Rename column from person_id to `target` in retention queries
No behavioral change, preparing for groups work :)
* Remove dead if statement
* WIP: Retention aggregation by groups
* Handle aggregation by groups in retention
Also handles the case where not every event has a property defined
* Test groups validation mixin
* Reformat
* Improve test for aggregation in retention
* Add /api/projects/@current/groups/property_definitions
To load property definitions for groups
* Add routes for fetching group types
* FE: Group analytics feature flag
* FE: Group analytics base types for properties
* FE: Initial implementation of fetching logics
* add group tabs to filter
* make sure group properties populate and refactor things
* prettier
* disable unused vars
* allow indexing objects by number
* mount group properties logic
Co-authored-by: Li Yi Yu <li@posthog.com>
* Extract GroupsJoinQuery
* Add test for breakdown filtering
* Unify breakdown mixins
* Allow passing breakdown_type == 'group' with breakdown_group_type_index
* Allow breakdown by group props in trends
* Add tests for trends breakdown_props function on group breakdowns
* Solve common issues
* Output snapshot diff into console
* Clean up materialized columns after tests
* Add zero protection
* Solve test failure
* Type math in Entity
* Allow passing group_type_index from FE to BE
* Get a initial query running
* Add group value filter if aggregating by groups
* Add snapshot testing for trends queries
* isort
* Update tests
* Add test for column_optimizer
* Update ee/clickhouse/queries/trends/util.py
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* adds a test to prove that without any changes the API receives a string and returns one see #938
* put a guard in place so that funnel query can be shifted to arrays without affecting existing clients
* make checking for single property breakdowns safer
* sort imports
* sort imports
* reformat files
* Revert "reformat files"
This reverts commit 58530f134f191d53b927abe6fb6f5d2d384cb255.
* expand short variable names
* corrects a typo
* it wasn't a typo it was a logic error
* Add group type, group_type_index
* Raise an error when handling unsupported properties in CH
* Improve repr
* Fix is_superset function
This was previously broken - sorting and zipping doesn't really work for
this intent.
* Add group_type_index to analysis results
* Add `group_types_to_query`
* Minor typing fixes
* Create groups tables in tests
* Simple first filter by groups query
* isort
* Use snapshot testing in event_query tests, add test for groups
Co-authored-by: Marcus Hyett (PostHog) <85295485+marcushyett-ph@users.noreply.github.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* feat(correlation): add team wide person property exclusion list
This change just adds the exclusion list to the `Team` model and checks
that it can be updated. Separately we can:
1. add interface for the Project page to display
2. pull and update this list from the funnels correlation page
NOTE: We don't perform any validation on the structure, :fingerscrossed:
this won't be an issue. Alternative would be to use ArrayField, but use
of JSONField is consistent with other fields.
* test(correlation): add test checking exclude properties pulled from team
* feat(correlation): load exclude properties from team settings
Previously we were loading exclude properties from local storage, so we
would not be sharing the exclusion list between users. This change
simply plugs the `excludePropertyNames` into the `teamLogic` for the
purpose of persisting and loading the values.
* use [teamLogic.actionTypes.loadCurrentTeamSuccess]
* get tests passing
* switch frontend to using `correlation_config`
* refactor: use Team.correlation_config for persistence
* add project settings for correlation
* fix lint
* ensure excluded properties are saved to project config
* Add default excluded properties
* format
* make propertyCorrelations not null
* ensure excluded property names config is unique
* rename excludeProperty to excludePropertyFromProject
* update var names
* change to targetProperties
* remove null special casing
* update to filter client side on exclude from project clicked
* update test name to reflect new functionality
* fix tests
Without this, to run plugin-server tests you need to reset all
containers every time since otherwise both test- and non-test clickhouse
would attempt to read from the same topic.
* backend fixes and test
* add breakdown value to pie chart
* adjust test
* fix faulty test
* fill param
* fix formula tests
* more date passing
* more cleanup
* all tests working
* make test data explicit and add better checks
* support both ee and postgres
* length checks
* paginate recording compression
* some tests
* more accurate duration calculation
* add tests and types
* tons of decompression fixes
* rename test file to avoid conflict
* move decompression to helper
* add test for helper
* type fix
* rename method
* simplify paginated decomression
* handle case where offset exceeds length
* clean up
* test fixes
* clean up on aisle 12
* Add surrounding object for metadata response
* Add table for group_type_mapping
* Remove materialized columns from events table schema
These are not used and not needed w/ new mat columns work
* WIP: Migration to add group analytics columns
* Remove event table changes temporarily
* initial refactoring
* popup UI
* refactor path cleaning logic
* add nullable
* all ui working
* fix migration
* use regex replacement from team object
* add flag
* add switch
* fix type
* fix type
* UI update
* restore removed arg
* add local path cleaning filters to api
* add test for local path filters
* working new UI
* reduced repeated code
* fix numbering
* minor refactoring
* update copy
* add under advanced features
* address comments, minor cleanup
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
Previously we would attempt to generate a response even though there
were no steps. There appears to be some code paths that blow up if this
happens, so instead we return as soon as we can in this case. This
appears to be the behaviour elsewhere also.
This resolves the sentry error found here:
https://sentry.io/organizations/posthog/issues/2718768248/
* paginate session recording events api
* code quality
* don't rerender replayer every time events get loaded
* refactor session recordings, make it backwards compatible, add bunch of tests
* move limit and offset to after decompress
* change limit
* add caching of recording
* add duration
* fix a few tests
* fix api tests
* add partial chunk test
* fix default limit test
* code quality
* typing
* fix backend tests
* mypy fixes and signature
* remove circular dependency
* mypy and sessionsplaylogic tests
* fix migration
* make single migration
* revert mypy typing
* remove require react resolve
* fix loading state to persist across multi chunks and add test'
* cleanup
* duration to ms
* fix duration tests
* remove require react resolve
* fix test
* bump rrweb player
* change cache behavior plus more
* fix frontend tests and make duration calculation more robust
* fix timestamp in ms bug in tests
* fix duration 0 test
* fix compatibility with rrweb-player
* update lockfile
* yarn unlink
* resolve kea-test-utils changes
* another one
* remove caching for now (leave as todo) and change limit to 100
