* add recordings to path query
* uncomment cache
* add clarifying comment
* works for start/end paths
* move to extra fields/properties
* add tests
* cleanup
* update ff name
* fix flaky test
* test and handle path_dropoff_key case
* refactor(lifecycle): simplify clickhouse sql logic
This updates the SQL to be comprised of two queries, one for getting
new, returning, and resurrecting periods of activity, one for getting
dormant periods right after periods of activity.
Refers to https://github.com/PostHog/posthog/issues/7382
* refactor(lifecyle): use `ClickhouseEventQuery` to build event query
* format
* Use bounded_person_activity_by_period for both sides of dormant join
* refactor(lifecycle): reduce pdi2 join by one
This means we're now under the current query memory limit for orgs with
around 20m distinct_ids. It does remove some readability though :(
* update snapshot
* Add further comments to query
* Add further comments to query
* Add further comments to query
* Remove dead variables
* Refactor person_query overriding
* Lifecycle refactoring continued
* Update lifecycle tests (except people ones)
* Make lifecycle people endpoint happy
* Remove django lifecycle tests
* Add some edge case tests
* Add missing type
Co-authored-by: Harry Waye <harry@posthog.com>
* Resolve weird SQL formatting issue
* Use a discriminated union for ActorType
* Use standard response types for related groups
* Update typing
* Always filter related actors by group type index
* Update snapshots & typing
* add trends option to experiment
(cherry picked from commit be9b8911f9)
* connect trends experiments
* clean up some stuff
* fix test
* results chart display for trends
* fix some issues with trends
* add in loading states and different insight types results wording
* move experiment results into loader
* add progress bar and clean up loading state
* fix experiment insight type bug
* add text beneath percent bar
* typing..
* single filter hide duplicate row button
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* Set time bounds for "all of time" filter
We won't display data points from before 2015 anymore, avoiding
confusion like in https://github.com/PostHog/posthog/issues/7626
* Disable dates before 2015, add tooltip
* Remove `minute` interval support from frontend
* Remove minute support from backend, default to hour instead
Also adds tests for interval logic
* Remove now-dead minute support from backend
* revert interval_candidate logic
* Move IntervalMixin to separate file
* Run queries against person_distinct_id2 when async migration is done
* Only write to clickhouse_person_unique_id topic if async migration is incomplete
* Update query snapshots
* Update plugin-server
* Adjust caching logic
* chore(lifecycle): add comments and CTEs to clickhouse sql
It was really big and confusing, but hopefully this clarifies a little
what is going on. As a followup PR I'll be doing some work to make the
query faster :fingerscrossed: but I think worth at least getting this
in, assuming I haven't broken any tests!
* update snapshots
* remove day references
* update snapshots
* fix(retention): add regression test for not_icontains filter
Previously if using the not_icontains filter, we were trying to double
substitute a query when fetching persons for retention. This adds a
test (and in the next commit a fix) such that we instead apply the
clickhouse substitute only once to each part of the query.
Resolves https://github.com/PostHog/posthog/issues/7747
* avoid double substitution
* Rework solution to not change ActorBaseQuery interface
* Remove unued import
* fix typing
* fix(retention): fix breakdown people urls
This change returns people_url for each breakdown cohort in the
response. We also merge the initial and returning queries together,
as this makes it easier to align the people query also.
Note that I'm talking about person_id as opposed to actor_type etc.
but perhaps that can be a followup.
* clean up clickhouse params
* tidy up a little
* remove import
* remove non-breakdown specific code
* make cohort by initial event date a special breakdown case
* keep date for backwards compat
* Remove unused sql
* make test stable
* wip
* Get most of the tests working
* test(retention): remove graph retention test
We no longer need this, we have all the information we need from the
table response for retention, and can construct this on the frontend.
* revert any changes to posthog/queries/retention.py
* revert any changes to ee/clickhouse/models/person.py
* Revert posthog/queries/retention.py to merge-base
* Ensure actor id is a str
* Add type for actor serialiser for type narrowing
* run black
* sort imports
* Remove retention_actors.py
* fix typings
* format
* reverse str type
* sort imports
* rename
* split out functions
* remove deuplicate logic
* working
* fix type
* don't stringify
* fix test
* ordering doesn't matter
* trigger ci
Co-authored-by: eric <eeoneric@gmail.com>
* error if unsupported db version
* upgrade mypy
* fix various types for mypy
* you can have it in any color you want, as long as it's black
* fix mypy
* Update `kafka-python`
* Format with Black
* Fix mypy after merge
Co-authored-by: Michael Matloka <dev@twixes.com>
* convert to actor form
* change var name
* remove unused imports
* typing issue
* use subquery
* bad import
* groups for general retention query
* actor in period
* update imports
* update test
* remove comment
* Create a new way to get distinct id queries thats gated by team_id
* Update most cases to use the new query
* Convert EVENT_JOIN_PERSON_SQL to new query
* Mostly convert GET_DISTINCT_IDS_BY_PERSON_ID_FILTER
* Mostly convert GET_DISTINCT_IDS_BY_PROPERTY_SQL
* Convert GET_PERSON_IDS_BY_FILTER
* Flag benchmarks
* Resolve circular imports
* Update a snapshot test
* Add a test for the new query logic
* test(retention): add http api tests for retention breakdowns
This just adds tests for person and event property breakdowns
* Add helper function for substituting clickhouse sql params
* feat(retention): add support for multiple breakdown props
This just reuses the work done for funnel multiple breakdown values. I
haven't tested this with anything other than person or event properties.
Rather than try to get it working for all the other property types.
The change adds a new `breakdowns` parameter to the retention endpoint,
that is the same as for funnels, e.g. it looks like:
```
{
...
"breakdowns": [{"type": "person", "property": "os"}, ...]
...
}
```
The return structure is the same as the non-breakdown version, except we
also include a `breakdown_values` property that is e.g. `["Chrome",
"95"]`, and the `label` attribute for this case would be "Chrome::95".
* fix typing
* update query snapshots
* remove unused imports
* use `self.assertEqual` instead of `assert`
* Remove explicit should_join_persons
* Revert limit by changes, use breakdown_type
* update snapshots
* fix typing
* use json_encode_request_params in retention test requests
* update json_encode_request_params to encode_get_request_params to better reflect purpose
* perf(retention): add benchmark query for retention with breakdown
* no materialize
* Use GroupTypeIndex type in BE
* Consistent ordering for group types
* Better BE type for group type index
* Handle non-url safe keys in groups
Previously this would crash due to never decoding the url and not
handling it in the backend
* Allow fetching related groups from the API
* Solve minor type errors
* Add a tooltip
* data-attr for tracking links
* Add related groups onto person page
* Fix an import
* Kill useEffect
