* ColumnOptimizer: Add functionality for person materialized columns
* WIP: Use person properties in trends/funnels
* Test person materialized props in trends cohort breakdown query
* Make use of materialized person properties in breakdowns
* Mark some cases working with materialized columns
* Test and fix breakdown by person props without filtering
* Make filtering by entity/person props on a join work better
By not assuming everything is under `properties`
* Add test case around breakdown with person properties
* Add test cases around materialization
* Add another materialization test
* Test cohorts and fix breakdowns with person filter
This exposes a limitation in the current implementation
* Fix some cohort tests
* Fix event query tests
* Get a funnel materialized column test running
* Cover more funnel breakdown tests with materialized
* Handle person property breakdowns in funnels
* Fixup funnel typo
* Add tests, fix an indentation issue
* TestFunnelPersons with materialized columns
* Test funnels against actions with person filters
* Add failing test for entity filtering failing
* Add test case for filtering with entity properties
* Show my 'broken' test is actually doing a subquery
* Resolve linting issues
Looking at the generated SQL it has bothered me that the list of entity
names is not unique for funnels in queries like `event IN [somearray]`
This tries to fix that.
* Speed up EE tests by not destroying tables
* Fix broken tests
* Speed up tests by not running migrations
* Fix last test
* req file?
* caching
* Parallel
* revert exp
* debug without parallel
* Revert "debug without parallel"
This reverts commit 83b3ad633f.
* Undo parallel tests
* Speed up more tests
* use final
* correct final
* print exception
* more prints
* move test
* add filter
* remove duplicate test
* prints here
* test wait
* set variable in query
* remove mutations_sync
* cleanup
Co-authored-by: eric <eeoneric@gmail.com>
* Remove dead argument
* Make allow_denormalized_props always explicit
* Change prop_clauses default
* Create a testing decorator for checking materialized columns
This makes it easier to have proper test coverage for materialized
columns and make sure no bugs creep up :)
* Fix event_query
* Test more materialized columns in trends
* Add materialized column tests for funnels
* Cleanup path_event_query
* Fix default
* Fix issue with clashing parameter names
* Fix fetching event-based breakdown values with `LIMIT`
* Improve test comment
* Limit prop clauses inclusion to breakdown queries
* Address feedback partly
* Fix imports
* Remove non-3.7-compatible `Literal`
* Don't try to import `Literal`
* Fix oversight
* Always include prop filters in `get_entity_filtering_params`
* Revert "Always include prop filters in `get_entity_filtering_params`"
This reverts commit 4f61edd19a.
* Add and test handling of actions in addition to events
* Update test_funnel.py
* Move new tests to `breakdown_cases`
* Cache action lookups on entity
* Avoid mutating filters
* Single-line a long query line
* Calculate columns to select from table dynamically
* Rename class to ColumnOptimizer
* Improve typing
* Set logic
* Rename var
* Fetch funnel top values via materialized columns
* Add a cast
* Simplify, fix
* Move types to more logical place
* Add tests for ColumnOptimizer
* Tidy code
* Test whether elements_chain is needed or not
* No caching in tests
* Use `ColumnOptimizer` in trends
* Prepend commas
* Fix event_query tests
* Handle another edge case around elements_chain
* Handle funnel exclusions
* Add documentation around trickier pieces of logic
* Comment out a broken case for now
* Extract method from funnels code
* Add failing test case for using denormalized property in a funnel breakdown
* Handle materialized event properties in funnel breakdown
* Refactor property lookups
* Use denormalized properties in paths
* Update validation tests
* Update validation tests
* Fix a broken test
* Move test around
* allow cohort breakdown to show
* Sanitize `order` in `Entity.__init__`
* Remove kludge from #5485 by making `entity.order` useful in practice
* Use synthetic `index` instead of user-provided `order`
* Fix `processed_entities` sorting
* Update frontend/src/scenes/funnels/funnelLogic.ts
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
* change type guard and add number back
* remove typeguard and add correct types for cohorts
* remove type change
* working cohort breakdowns ui
* one more type fix
* adjust tests
* adjust tests
* more test adjustmenet
* minor syntax change
* use breakdown type
* add comment
* remove leakover
Co-authored-by: Michael Matloka <dev@twixes.com>
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
Co-authored-by: Sam Winslow <sammywinslow@gmail.com>
* Add migration marking existing columns as materialized with a comment
* Add method to allow looking up materialized columns
* Add test for fetching and creating materialized columns
* Disable caching in tests
* Remove some callsites to CLICKHOUSE_DENORMALIZED_PROPERTIES
* Fixup caching logic
* Add (currently broken) test for denormalizing person properties
* Update funnel event query to work with new functionality
* Update other tests/queries
* Remove now-unused setting
* Use proper interval calculation in the funnel trends query
* Add some comments
* Update `test_filter`
* Rework `NULL_SQL` to use CH `INTERVAL` too
* Fix week-based relative `date_from` support not existing
* Make use of `toInterval*` functions and inject less
* Add fallback for `date_from` in `ClickhouseSessionsAvg`
* Improve cached_property typing
Noticed that e.g. `filter.breakdown` was getting inferred to be `Any`
which is not correct. Added generics to fix it :)
* Proposed fix: make filter_test_accounts return a bool always
* Fix warning in clickhouse_sessions.py
* Cast in session.events
* Cast 2x in funnel queries
* Ignore error in session recording
We know the valid values here
* Add assertions in stickiness filters
* Cast in more funnel queries
* Untyped dict where inferred type is wrong
* Add types to abstract methods
* Type prop_vals
* Add a lot of casts
These are correct. We should really validate while parsing instead
* Add more casts to funnel trends
* Last fixes
* add more options for funnel windowing
* tweaks
* add test
* accomodate any case and make to_dict smaller
* remove import
* change name
* store lower
* fix test
* partial implementation
* working event breakdown
* remove hardcoded fill
* add person test
* remove test unneeded
* use original breakdown implementation
* add cohort handling
* add clause lost on merge
* add median time for conversion steps
* patch tests
* fix time to convert funnel query
* fix strict and unordered
* add median conversion time to base
* fix
* more tests
* fix broken funnel pagination
* account for both return results
* add limit param
* move order
* move the check
* change magic number
* adjust more tests
* add all steps option
* all steps working; add total and mean time to convert
* change display type checks to use enum
* kea types
* dangling console log
* Add average conversion time to time to convert results
* respond to feedabck
* responsive histogram sizes
* merged @Twixes backend changes; adjust data shape on frontend; add responsiveness to histogram
* add tooltip label
* adjust copy and tooltip
* minor tweaks
* respond to general feedback
* kea auto
* better empty state:
* error handling null time bins
* fix tests
Co-authored-by: Michael Matloka <dev@twixes.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* JOIN in funnel trends query in a more optimal way
* Reformat funnel trends assert slightly
* Refactor `ClickhouseFunnelTrends` slightly
* Add `ClickhouseFunnelTrendsPersons` code
* Allow parametrization of funnel trends persons query
* Update test_funnel_trends.py
* Clean some funnel analysis code up
* Fix `drop_off` default in `FunnelTrendsPersonsMixin`
* Refactor for `ClickhouseFunnelTrends.get_step_counts_without_aggregation_query`
* Update test_funnel_trends.py
* Add an API test for funnel trends persons
* Use `FUNNEL_PERSONS_BY_STEP_SQL`
* Tests persons some more
* Remove unused imports
* first pass mix and match everything
* cleanup
* small refactoring of get_query
* remove top level filter props
* save the file
* enable person breakdowns querying for regular + strict funnels
* move mixin testing to more appropriate location
* add unordered, testing quality of life improvements
* clean up
* add strict+unordered connectivity test for time to convert
* trim after extracting
* clean up, fix tests, trim everywhere
