* fix(retention): fix breakdown people urls
This change returns people_url for each breakdown cohort in the
response. We also merge the initial and returning queries together,
as this makes it easier to align the people query also.
Note that I'm talking about person_id as opposed to actor_type etc.
but perhaps that can be a followup.
* clean up clickhouse params
* tidy up a little
* remove import
* remove non-breakdown specific code
* make cohort by initial event date a special breakdown case
* keep date for backwards compat
* Remove unused sql
* make test stable
* wip
* Get most of the tests working
* test(retention): remove graph retention test
We no longer need this, we have all the information we need from the
table response for retention, and can construct this on the frontend.
* revert any changes to posthog/queries/retention.py
* revert any changes to ee/clickhouse/models/person.py
* Revert posthog/queries/retention.py to merge-base
* Ensure actor id is a str
* Add type for actor serialiser for type narrowing
* run black
* sort imports
* Remove retention_actors.py
* fix typings
* format
* reverse str type
* sort imports
* rename
* split out functions
* remove deuplicate logic
* working
* fix type
* don't stringify
* fix test
* ordering doesn't matter
* trigger ci
Co-authored-by: eric <eeoneric@gmail.com>
* error if unsupported db version
* upgrade mypy
* fix various types for mypy
* you can have it in any color you want, as long as it's black
* fix mypy
* Update `kafka-python`
* Format with Black
* Fix mypy after merge
Co-authored-by: Michael Matloka <dev@twixes.com>
* convert to actor form
* change var name
* remove unused imports
* typing issue
* use subquery
* bad import
* groups for general retention query
* actor in period
* update imports
* update test
* remove comment
* test(retention): add http api tests for retention breakdowns
This just adds tests for person and event property breakdowns
* Add helper function for substituting clickhouse sql params
* feat(retention): add support for multiple breakdown props
This just reuses the work done for funnel multiple breakdown values. I
haven't tested this with anything other than person or event properties.
Rather than try to get it working for all the other property types.
The change adds a new `breakdowns` parameter to the retention endpoint,
that is the same as for funnels, e.g. it looks like:
```
{
...
"breakdowns": [{"type": "person", "property": "os"}, ...]
...
}
```
The return structure is the same as the non-breakdown version, except we
also include a `breakdown_values` property that is e.g. `["Chrome",
"95"]`, and the `label` attribute for this case would be "Chrome::95".
* fix typing
* update query snapshots
* remove unused imports
* use `self.assertEqual` instead of `assert`
* Remove explicit should_join_persons
* Revert limit by changes, use breakdown_type
* update snapshots
* fix typing
* use json_encode_request_params in retention test requests
* update json_encode_request_params to encode_get_request_params to better reflect purpose
* perf(retention): add benchmark query for retention with breakdown
* no materialize
* working for unique_groups math
* fix types
* add null check
* update snapshots
* update payload
* update snapshots
* use constructor
* adjust queries
* introduce base class
* consolidate querying
* shared serializer and typed
* sort imports
* snapshots
* typing
* change name
* Add group model
```sql
BEGIN;
--
-- Create model Group
--
CREATE TABLE "posthog_group" ("id" serial NOT NULL PRIMARY KEY, "group_key" varchar(400) NOT NULL, "group_type_index" integer NOT NULL, "group_properties" jsonb NOT NULL, "created_at" timestamp with time zone NOT NULL, "properties_last_updated_at" jsonb NOT NULL, "properties_last_operation" jsonb NOT NULL, "version" bigint NOT NULL, "team_id" integer NOT NULL);
--
-- Create constraint unique team_id/group_key/group_type_index combo on model group
--
ALTER TABLE "posthog_group" ADD CONSTRAINT "unique team_id/group_key/group_type_index combo" UNIQUE ("team_id", "group_key", "group_type_index");
ALTER TABLE "posthog_group" ADD CONSTRAINT "posthog_group_team_id_b3aed896_fk_posthog_team_id" FOREIGN KEY ("team_id") REFERENCES "posthog_team" ("id") DEFERRABLE INITIALLY DEFERRED;
CREATE INDEX "posthog_group_team_id_b3aed896" ON "posthog_group" ("team_id");
COMMIT;
```
* Remove a dead import
* Improve typing for groups
* Make groups updating more generic, avoid mutation
This simplifies using the same logic for groups
Note there's a behavioral change: We don't produce a new kafka message
if nothing has been updated anymore.
* Rename a function
* WIP: Handle group property updates
... by storing them in postgres
Uses identical pattern to person property updates, except we handle
first-seen case within updates as well.
* Get rid of boolean option
* WIP continued
* fetchGroup() and upsertGroup()
* Test more edge cases
* Add tests for upsertGroup() in properties-updater
* Rename to PropertyUpdateOperation
* Followup
* Solve typing issues
* changed implementation to use pg
* unusd
* update type
* update snapshots
* rename and remove inlining
* restore bad merge code
* adjust types
* add flag
* remove var
* misnamed
* change to uuid
* make sure to use string when passing result
* remove from columnoptimizer logic and have group join logic implemented by event query classes per insight
* remove unnecessary logic
* typing
* remove dead imports
* remove verbosity
* update snapshots
* typos
* remove signals
* remove plugin excess
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
* Add groups stuff
* Rename column from person_id to `target` in retention queries
No behavioral change, preparing for groups work :)
* Remove dead if statement
* WIP: Retention aggregation by groups
* Handle aggregation by groups in retention
Also handles the case where not every event has a property defined
* Test groups validation mixin
* Reformat
* Improve test for aggregation in retention
* Refactor column_optimizer to work differently
* WIP: Use counter over set
* Handle person filters in person query
* Remove a dead argument
* Use enum over parameter for determining behavior
* Allow excluding person properties mode when handled in person query
* Fix _get_person_query type
* Use correct table for funnel_event_query
* Remove unneeded override
* Add extra typing
* Filter by entity.properties in person query for trends
* Handle error 184 due to naming clash
* Better default for prop_filter_json_extract
* Update column_optimizer tests for Counter
* Handle person_props as extra_fields
* Handle breakdowns and person property filter pushdown
* Transform values correctly
* Simplify get_entity_filtering_params
* Fix funnel correlations
* Solve caching issues in trend people queries
* Remove @skip test
* Add syrupy tests for parse_prop_clauses
Can update these via --snapshot-update
* Add snapshot tests for person queries
* Add a few notes
* Update test to avoid collision
* Kill dead code
* Handle PR comments
* Update ee/clickhouse/queries/person_query.py
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
