0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 18:26:15 +01:00
Commit Graph

13 Commits

Author SHA1 Message Date
Karl-Aksel Puulmann
3a4beda2ad
Speed up person_distinct_id queries for select clients (#7577)
* Create a new way to get distinct id queries thats gated by team_id

* Update most cases to use the new query

* Convert EVENT_JOIN_PERSON_SQL to new query

* Mostly convert GET_DISTINCT_IDS_BY_PERSON_ID_FILTER

* Mostly convert GET_DISTINCT_IDS_BY_PROPERTY_SQL

* Convert GET_PERSON_IDS_BY_FILTER

* Flag benchmarks

* Resolve circular imports

* Update a snapshot test

* Add a test for the new query logic
2021-12-08 15:59:41 +02:00
Tim Glaser
7fe96b813d
Fix error too deep recursion (#7445)
* Fix error too deep recursion

* Add space

* space no 'AND'

* update snapshots

* fix test

* fix test

* Remove team_id from prop_clause
2021-12-02 12:12:14 +01:00
Rick Marron
6fc64efafa
Only include events in session recording query when needed (#7248)
* query for events with matching session_id if possible

* move test to clickhouse

* move event list to list instead of set for flaky test snapshot

* join events only when needed

* trigger benchmarking

* typo
2021-11-30 09:33:41 -08:00
Rick Marron
fd150761ec
Materialize has_full_snapshot in session_recording_events (#7281)
* materialize has_full_snapshot

* add basic backfill command

* some clean up

* bug fix

* use comment function

* remove unneeded column check

* add to test for get_materialized_columns

* update test

* ClickhouseDestroyTablesMixin resets session_recording_events

* remove backfill command
2021-11-29 16:18:08 -08:00
Rick Marron
2c417ed1e1
Use session_id in recordings query when possible (#7218)
* query for events with matching session_id if possible

* move test to clickhouse

* move event list to list instead of set for flaky test snapshot

* comment to clarify test
2021-11-22 13:23:08 -08:00
Rick Marron
9b9b12088a
Add 'active segments' to recording metadata endpoint (#7179)
* remove redundant query

* add include_active_segments query parameter to metadata endpoint

* bug fix

* type fixes

* respond to feedback

* add back in query snapshot
2021-11-18 11:32:10 -08:00
Rick Marron
c55e834bc1
include elements chain in recordings query when needed (#7147) 2021-11-15 18:35:45 -08:00
Rick Marron
60545e9c93
Recording query improvements (#7006)
* seperate clickhouse and postgres queries

* move to single query

* remove ununsed imports

* fix cohort + subquery issues

* some cleanup

* fix mypy issue

* fixes to query + tests

* add test for person and cohort properties

* remove cruft

* fix api tests

* Remove cohort + person prop filter from postgres

* move cohort and person property tests to clickhouse

* Filter down events

* typing fix

* fix test

* remove unneeded snapshot

* tests working with snapshots

* switch limit to property

* use materialized columns

* move to new distinct_id query

* add snapshots

* fix materialized column bug

* support cohort and person prop filters

* add person and cohort filtering

* remove cohort filtering from clickhouse

* small tweaks

* Lint fix
2021-11-15 16:15:37 -08:00
Rick Marron
f2076f4008
Paginate recording compression (#6563)
* paginate recording compression

* some tests

* more accurate duration calculation

* add tests and types

* tons of decompression fixes

* rename test file to avoid conflict

* move decompression to helper

* add test for helper

* type fix

* rename method

* simplify paginated decomression

* handle case where offset exceeds length

* clean up

* test fixes

* clean up on aisle 12

* Add surrounding object for metadata response
2021-10-25 12:59:54 -07:00
Alex Gyujin Kim
074d633c53
Make Sessions Recording faster!! (#6270)
* paginate session recording events api

* code quality

* don't rerender replayer every time events get loaded

* refactor session recordings, make it backwards compatible, add bunch of tests

* move limit and offset to after decompress

* change limit

* add caching of recording

* add duration

* fix a few tests

* fix api tests

* add partial chunk test

* fix default limit test

* code quality

* typing

* fix backend tests

* mypy fixes and signature

* remove circular dependency

* mypy and sessionsplaylogic tests

* fix migration

* make single migration

* revert mypy typing

* remove require react resolve

* fix loading state to persist across multi chunks and add test'

* cleanup

* duration to ms

* fix duration tests

* remove require react resolve

* fix test

* bump rrweb player

* change cache behavior plus more

* fix frontend tests and make duration calculation more robust

* fix timestamp in ms bug in tests

* fix duration 0 test

* fix compatibility with rrweb-player

* update lockfile

* yarn unlink

* resolve kea-test-utils changes

* another one

* remove caching for now (leave as todo) and change limit to 100
2021-10-15 02:41:47 +00:00
Karl-Aksel Puulmann
457e151f58
Push person predicates down (#6346)
* Refactor column_optimizer to work differently

* WIP: Use counter over set

* Handle person filters in person query

* Remove a dead argument

* Use enum over parameter for determining behavior

* Allow excluding person properties mode when handled in person query

* Fix _get_person_query type

* Use correct table for funnel_event_query

* Remove unneeded override

* Add extra typing

* Filter by entity.properties in person query for trends

* Handle error 184 due to naming clash

* Better default for prop_filter_json_extract

* Update column_optimizer tests for Counter

* Handle person_props as extra_fields

* Handle breakdowns and person property filter pushdown

* Transform values correctly

* Simplify get_entity_filtering_params

* Fix funnel correlations

* Solve caching issues in trend people queries

* Remove @skip test

* Add syrupy tests for parse_prop_clauses

Can update these via --snapshot-update

* Add snapshot tests for person queries

* Add a few notes

* Update test to avoid collision

* Kill dead code

* Handle PR comments

* Update ee/clickhouse/queries/person_query.py

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
2021-10-13 14:00:47 +00:00
Rick Marron
63d127df13
Filter session recordings (#6162)
* fix bug where session recording is split by new distinct_id

* fix query bug to allow distinct_id clause

* Basic query filtering by events on clickhouse

* add duration filtering

* Add multiple action/event filters

* Move to using person_id + optimize query

* add postgres support and querying on person_id

* move back to distinct_ids

* remove lots of duplicated logic

* Add pagination to backend

* Pagination

* Add duration and date filter

* Use person header is list view

* type fixes

* bug fix

* bugs and tests

* test fixes

* style filters

* logic tests

* Add another test for team leakage

* fix flaky test

* UX changes

* UX tweaks

* auto focus filter box on open

* add test for recording viewed change

* move duration filter to using a tested logic

* move to clsx

* add debounce breakpoint

* Small style change

* fix debounce

* fix breakpoint

* move test to defaultAPIMocks (merging changes)

* react to url change
2021-10-11 13:15:05 -07:00
Rick Marron
a0775d7fae
Replace sessions with session recordings (#6050)
* Move Sessions tab -> Session Recording (behind feature flag)

Note: the /scenes/sessionRecordings folder is a copy of /scenes/sessions. Planning to remove sessions from the new SessionRecordings folder over the next coupld of commits

* Add very basic session recording page

* Add session recordings to persons page and session recs can play

* add endpoints for clickhouse

* fix ordering in existing tests

* add basic test to pg and clickhouse

* small bug fix

* type fix

* add test for data leaking between teams

* fix navigation bugs

* Fix name of logic

* Add basic logic tests

* fix tests

* fix laoding state bug

* revert changes to existing session queries

* move to proper queries for session recordings

* type fixes

* rename feature flag

* type fix

* Fix capitalization

* remove showSessions and clean up ifs

* Style changes and refactor SessionPlayerDrawer

* bug fix

* move to using query params

* remove type from persons logic

* Move sessionRecordingTableLogic key to distinctId

* rename setSessionRecordingId to openSessionPlayer

* improve tests

* Cleanup from review

* move team and limit to parameters
2021-09-28 11:18:38 -07:00