* WIP persons table schema migration
* add version to create_person
* test increased timeout
* Update ee/clickhouse/migrations/0019_person_table_version_column.py
* remove flaky kafka test
* format
* test a new approach - make this a non-large migration?
* Update ee/clickhouse/migrations/0021_person_table_version_column.py
* update
* Refactor stickiness to have its own event_query
This will speed up queries significantly and allow for filtering by
group properties
* Use same event_query for stickiness people
* Minor cleanup
* Add tests (and missing file) to group filtering in stickiness
* Allow aggregating by groups in stickiness
* Show group property filters in FE for stickiness
* convert the smallest of the funnel test files
* convert unorder funnel person test to journey for helper
* convert another file
* convert another file
* convert another file
* convert another file
* convert another file
* undelete snapshot files
* undelete snapshot files
* Revert "convert another file"
This reverts commit ef08511509.
* seperate clickhouse and postgres queries
* move to single query
* remove ununsed imports
* fix cohort + subquery issues
* some cleanup
* fix mypy issue
* fixes to query + tests
* add test for person and cohort properties
* remove cruft
* fix api tests
* Remove cohort + person prop filter from postgres
* move cohort and person property tests to clickhouse
* Filter down events
* typing fix
* fix test
* remove unneeded snapshot
* tests working with snapshots
* switch limit to property
* use materialized columns
* move to new distinct_id query
* add snapshots
* fix materialized column bug
* support cohort and person prop filters
* add person and cohort filtering
* remove cohort filtering from clickhouse
* small tweaks
* Lint fix
* remove cohort comment
* feat(funnels): update frontend to use new people_urls in response
This change updates the `FunnelBarGraph` and `FunnelStepTable`
components to use the new `converted_people_url` and
`dropped_poeple_url` in the funnels response.
I need to check that this covers all the cases for funnels. I've only
added people urls for funnels of type step for ordered/unordered/strict
and haven't yet touched the time bins and conversion time funnel
endpoint variants.
* add /some/people/url/ to tests
* chore: add action back, update tests to reference it
* fix typing
* wip
* get list of groups
* move backend endpoints
* frontend wip
* fix routes for groups
* add groups file
* try to fix tab nav
* fix groups query
* reformat results, add limits and offsets and next url
* move groups pages logic into own file
* fix and clean up tabs
* account for pagination and response reformatting
* lint fixes
* optional prev input
* typescript errors
* typescript fixes
* address feedback comments
* groups list logic test
* render properties expandable row
* fix test and lints
* prettier
* simplify table column title
* Fix a typing issue
* Use enum > boolean parameter
This makes what's going on more explicit
* Turn off pagination at the bottom of the table
* Rename table column
* Make pagination in groups work
Quite bugs fixed:
1. `previous` page was always shown, even if going into negative offsets
2. query_result could never be larger than `limit` with the code
* Cleanup code
* Make sidenav active under persons
* Use the correct query
* Add tests for pagination
* Update wording again
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
* cumulative dau for non breakdown
* breakdown case
* clean up test
* add trends request util
* use simpler test util
* types
* insight tests
* add snapshots
* black
* adds a first draft of a test helper for clearer event setup in tests
* one spelling of journeys
* obey the type checker
* convert all of breakdown_cases to the new test helper
* Removes unused helper method
* test(funnel-people): add tests for converted/dropped people urls
As per https://github.com/PostHog/posthog/issues/6935 we are adding urls
to responses to attempt to improve consistency between insight responses
and the people we display in the UI.
This simply adds tests for these urls, just for funnel step response
shape. The other two types of response will be handled separately.
* feat(funnel-people): return converted/dropped people url in funnel
Here I have simply added converted / dropped urls in the base funnel.
This touches both clickhouse and postgres functionality, so I will
probably add in tests for postgres also, or just disable to postgres.
It doesn't pass yet as there is an issue in that the order appears to be
ignored by the people endpoint.
* fix(funnels): actually use strict/unordered persons querying
Previously we were always using the standard `ClickhouseFunnelPersons`
class for retrieving people from the `/api/persons/funnel/` endpoint.
This change selects from the unordered and strict variants based on the
`funnel_order_type` setting.
Refers to https://github.com/PostHog/posthog/issues/7058 although there
is a frontend component to add that will truely resolve the issue.
* feat(funnel-people): add people urls for funnels with breakdown
In this I have also removed any changes from the non-clickhouse code.
Note that there are two places I've added the url generation code, one
in the `ClickhouseFunnelBase` and another in the `ClickhouseFunnel`
I think the former covers the unordered and strict cases, and the later
for the breakdown case. I think there are further test that I'll need to
add to validate e.g. if breakdown + strict work as expected.
* Fix people urls for unordered funnels + breakdown
* test: add test for strict breakdown with people urls
* make funnel response assertions not check people url equality
* fix typing
* fix tests
* remove new line in postgres funnel.py
* clear cache on insight test start
* no really, clear the cache
* remove flakiness from strict funnel test
* correct the unordered test
* use absolute uris
* use step.index not step.order
* remove out of date comment
* use step.index, not step.order
* use step.index, remove unordered funnels comment
* use journeys_for instead of create_events
* add test_helper_methods
* move all the tests where the properties are the same for all events to the journey helper
* compare funnel results without caring about person order
* spell words correcterly
* Revert "spell words correcterly"
This reverts commit befb83b183.
* Revert "compare funnel results without caring about person order"
This reverts commit 268927b8ba.
* correct types for test props
* add groups property values
* update groups property tests
* newline
* fix
* update test
* python black
* clean up tests
* rename to endpoint in operator value select
* add empty property values test
* do not return in an array
* fix
* feat(correlation): use people_url to load people modal
This change adds a new method to the `personsModelLogic` to allow for
specifying a url to be used to retrieve people from. I've avoided
overloading the existing `loadPeople` and rather added a new one,
`lostPeopleFromUrl`.
Currenlty this is only being used by correlations, but the intention is
to extend it's usage to anywhere else that is performing drilldowns to
people from, for instance, the trends aggregation.
* add paycard back
* make labels look right
* remove unused code
* fix typing in test
* fix typing in personsModelLogic
* refactor to make eventUsageLogic work
* fix typing
* fix typing again
* remove some funnel stories
* get storybook working with paycard
* cleaned up the interface a little
* chore(correlation): use `api.get` for fetching people
This updates to use api.get, and also converts the returned uri from the
correlation endpoint to an absolut uri, instead of just an absolute
path.
The `api.get` method expects either an abolute uri, or a relative path.
I'd rather not use an relative path as it's not obvious what it should
be relative to.
* fix tests
* make storybook uris absolute
* Paths filtering by groups backend
* update correlation tests, now that CTEs are included in sqls
* use decorator for materialising to ensure clean up happens
* cleanup offending tests
* feat(correlation): add people drill down urls for event correlation
This change simply adds urls in the response of the correlation endpoint
for each correlation returned. Note that this only adds support when
using correlation_type=events, as the
correlation_type=events_with_properties requires further changes that
I'd like the separate out to keep this PR small.
* feat(correlations): add people by property to correlation response
This commit covers the PROPERTIES case for making sure we return a url
that can be used to retrieve the people associated with a correlation,
either success or failure. This case is a completely different url than
for the events people endpoint as we are reusing the funnel people
endpoint, and filtering by "funnel_step_breakdown".
I'm uncertain if this is actually the correct url at this point in time,
as I have just attempted to copy what I have seen in the request from
the live running app.
* fix test
* Remove another param
* fix mypy typing
* Fix tests
* Add url response support for event_with_properties
* remove type assertion
* remove variable hack
* Remove funnel_steps
* use with_data and to_params
* Remove comment regarding complexity, it seems to pass the tests so...
* Add autocapture support for people url
* refactor people url construction a little
* remove properties attr for event people request
* fix if
* no really, fix if not None
* Remove _assert_never, mypy needs updating
* Fix no return mypy error
* Remove json.dumps
* add window-id to ingestion
* move window_id into snapshot data
* Add tests
* add window_id to snapshot_data only if included in the event
* add window_id to session_recording_events table
* revert ph-js downgrade
* fix data generator
* remove partition change
* update window_id defaults for tests
* missed one window_id default
* rename migration for conflict
* Migration to use materialized columns for groups
Workaround for https://github.com/PostHog/posthog/issues/6422
* Use groups materialized columns in queries
* Update mat column creation tests
* Simplify aggregation_target_field
* Fix migration
* Update snapshots
* smallest change to make aggregation work
* address comments
* add snapshot
* move function to groups model
* update funnel snapshot
* rename person_id to aggregation_target
* update snapshots as well
* dont support persons query mods for now
* update snapshot
* make array orders deterministic
* Improve process_math
* Add test for overlapping group keys
* Improve event query tests
* Add test for filtering by person properties together with groups
* Avoid flaky tests due to cohort_id changing
* Update queries and snapshots
* Add groups stuff
* Rename column from person_id to `target` in retention queries
No behavioral change, preparing for groups work :)
* Remove dead if statement
* WIP: Retention aggregation by groups
* Handle aggregation by groups in retention
Also handles the case where not every event has a property defined
* Test groups validation mixin
* Reformat
* Improve test for aggregation in retention
* Add /api/projects/@current/groups/property_definitions
To load property definitions for groups
* Add routes for fetching group types
* FE: Group analytics feature flag
* FE: Group analytics base types for properties
* FE: Initial implementation of fetching logics
* add group tabs to filter
* make sure group properties populate and refactor things
* prettier
* disable unused vars
* allow indexing objects by number
* mount group properties logic
Co-authored-by: Li Yi Yu <li@posthog.com>
* Extract GroupsJoinQuery
* Add test for breakdown filtering
* Unify breakdown mixins
* Allow passing breakdown_type == 'group' with breakdown_group_type_index
* Allow breakdown by group props in trends
* Add tests for trends breakdown_props function on group breakdowns
* Solve common issues
* Output snapshot diff into console
* Clean up materialized columns after tests
* Add zero protection
* Solve test failure
* Type math in Entity
* Allow passing group_type_index from FE to BE
* Get a initial query running
* Add group value filter if aggregating by groups
* Add snapshot testing for trends queries
* isort
* Update tests
* Add test for column_optimizer
* Update ee/clickhouse/queries/trends/util.py
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
