* delete insight to dashboard relation on soft delete of either
* refactor tests
* obey mypy
* don't allow hard delete of insights or dashboards
* soft deleted insights return 404 on get
* prove soft delete can be reversed
* no need to test permissions on delete that nobody can do
* no need to test permissions on delete that nobody can do
* don't allow hard delete on cohort
* don't allow annotation deletion
* don't allow hard deletion of actions
* undelete snapshot :/
* don't allow hard deletion of feature flag
* soft delete feature flag deletes experiments
* Populate the new model
* Add constance to admin
* Handle table not existing
* Add model tests
* Add tests for the constance model
* Fixup migrations
* Update instance settings to use new constance model
* update most readings
* Use override method in some tests
* Improve typing
* Prettier
* Merge 233 migration
* Order imports
* Fixup
* Update some query counts
* Remove dead import
* Improve some mypy
* Update team.py
* Remove constance library
* Mark migration as non-atomic
* Instance settings cleanup
* Rename methods to be more consistent
* Fixup for migration
* Update tests
* add fields to property
* add validatoin
* fix naming'
* fix errors
* example
* example implementations
* remove none
* more typing
* change condition
* add terrible draft of lifecycle query
* move around date query
* one random test to satisfy
* add funnel persons subquery
* basic func
* use key as event
* change condition
* change to countif
* add base query conditions
* add comments
* condition building
* person props
* param cleanup
* basic test
* stub tests
* remove unnecessary funcs
* adjust typing
* wip
* add filters to cohorts
* add migration, start refactoring property types
* proof of concept of property refactor
* add migration too
* clean up property types
* slight more clean up, add comments
* removed optimizer
* add comment
* support performed_event_regularly
* remove unneeded
* sql params on regular query
* more tests
* typing
* some more clean up, enable person property push downs
* add snapshots too
* clean up types and validation
* fix typing
* remove unused snapshot
* fix bug
* Update ee/clickhouse/queries/cohort_query.py
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* oops
* update snapshots
* use materialized person props
* update fields
* add date value filter and tests
* event sequence join
* more join magic
* typing
* build fields
* see which tests fail
* resolve migrations conflict
* add some more tests
* fix first_time and event_restarted query
* add negation handling
* update snapshots
* add tests for multiple events case
* update stopped performing implementation
* regularly period validation
* old cohort support
* clean up + some tests
* fix some cohort tests issues
* remove migrations from this pr
* clean up
* support negations
* update snapshots
* more tests
* did all tests just pass?
* fix new failing tests
* fix tests, try new format for cohort insertions
* inter-test dependency resolve
* gracefully handle empty cohorts
* add type annotation
* address comments
* remove properties select in cohort removal query
* address comment
* update snapshots
* add freeze time
* fix some tests
* remove the Xs
* more test fixes and clean up
* raise on cyclic dependencies instead
* fixes
* test waters with parallel execution
* fixes
* update tests
* final test
* clean up
* more test fixes
* gahhhhh
Co-authored-by: eric <eeoneric@gmail.com>
Co-authored-by: Rick Marron <rcmarron@gmail.com>
* dashboards and tiles, view mode, basic frontend typing
* update types, many-to-many relations
* update insight dashboards
* duplication works
* mypy
* fix migrations
* fix duplication bug
* add default filters
* fix caching
* deprecation nonsense
* remove things
* add nplus1 tests
* make nplus1 test need no explanation
* don't need to prefetch and select related
* move layouts API onto dashboard and store updates on DashboardTile
* apply dashboard filters to insights when loading dashboard
* don't need to update cached result in a cached function
* ugh, filter caching
* update basic serializer test
* add defaults when getting layout and colour from insights
* caching insights depends on filters hash changing when the insight is saved
* caching insights depends on filters hash changing when the insight is saved
* can't constantly resave insight filter hashes any more
* prettify the skip comment
* fix patch reference
* fix test assertion
* need to save dashboards to update insight filters_hash *until we fix filtering*
* add comment explaining why the test is failing - so I remember later
* blunt solution to not overwriting dashboard aware filter hash
* explode by 7 instead of 9 queries per insight added to dashboard
* mark some learning as a TODO
* only update dashboard layout when in edit mode
* insight logic no longer deals with saving layout
* skip the nplus1 test _for now_
* update the dashboard snapshots
* enrich insights with extra props when loaded in dashboard context
* update color via dashboard not insight
* create empty insight with a dashboards array, not a single dashboard
* remove empty TODO
* avoid initialising dashboardsmodel for dashboards logic tests
* no need to select and prefetch related
* order migration by last modified at
* remove insight layouts endpoint
* Add new from_dashboard parameter to API docs
* typo
* move deprecated fields on insight
* fix syntax error in migration file
* add permissions tests
* add help text to dashboards serializer field
* fix test with silent server errors
* down to 3 extra queries per insight on each dasboard
* down to 1 extra queries per insight on each dashboard
* set dashboard on context when retrieving an insight, so that filters are reported correctly
* explicitly fail to add insight to another team's dashboard
* update insight cache in correct context when refreshing
* add migration test
* satisfy the type checker
* Revert "satisfy the type checker"
This reverts commit 4d48a16cf4.
* satisfy the type checker
* really satisfy the type checker
* address two nits
* can ignore conflicts when inserting dashboard tiles
Co-authored-by: Paul D'Ambra <paul.dambra@gmail.com>
* refactor(ingestion): establish setup for json consumption from kafka into clickhouse [nuke protobuf pt. 1]
* address review
* fix kafka table name across the board
* Update posthog/async_migrations/test/test_0004_replicated_schema.py
* run checks
* feat(persons-on-events): add required person and group columns to events table
* rename
* update snapshots
* address review
* Revert "update snapshots"
This reverts commit 63d7126e08.
* address review
* update snapshots
* update more snapshots
* use runpython
* update schemas
* update more queries
* some improvements :D
* fix naming
* fix breakdown prop name
* update snapshot
* fix naming
* fix ambiguous test
* fix queries'
* last bits
* fix typo to retrigger tests
* also handle kafka and mv tables in migration
* update snapshots
* drop tables if exists
Co-authored-by: eric <eeoneric@gmail.com>
* Add game engine base
Fix `breakdown_limit`
* Remove cruft
* Add global illumination
Recognize map as non-time series
* Add sprites
Make `breakdown_limit` automagical
* Fix raytracing
Add basic map to the frontend
* DirectX 12
* Fix things
* Fix things
* Add new game mechanics
* Feature-flag it
* Try fixing more snapshots
* Add TODOs
* Fix things further
* Add new skill tree
* Fix things at last
* Importing models from 3D Studio Max
* Revert side refactor
* Fix clipping bug
Fix table value display
* Refactor main game loop
* Fix warlock attack
* Fix spacing
* Flip tooltip to the left when there isn't enough space
* Switch away when there are multiple series
* Also hide legend for table
* Reveal the truth
* Refactor `countries` out into `countryVectors.tsx`
* Restore Kosovo
* Add a story
* Adjust mechanics of one-series limitation
* Optimize map
* Disable "Compare to previous" for map
* Optimize color scheme
Co-authored-by: Marcus Hyett (PostHog) <85295485+marcushyett-ph@users.noreply.github.com>
* feat(trends): add moving average `smoothing_intervals` param
This will average over the preceding `smoothing_intervals` intervals.
The driver for this is to remove either weekly or monthly trends.
Limitations at the moment are that it only works for total volume
queries, as it relies on the results from clickhouse being from a single
timeline series.
* WIP UI for selecting smoothing
* WIP: Fix typing and appearce logic
* Made smoothing extensible beyond day
* Test now considers a non-trivial case
* chore(smoothing): use url params for state storage of smoothing value
This should allow for url navigation to work, although I haven't quite
figured out how we then use this to make the insights/trends POST
request
* Added further tests for smoothing intervals
* fix test_to_dict
* fix trend tests
* Added test to validate preceeding limit
* Fixed test for 2 interval
* clear django redis cache before each test
* add explicit ordering to dashboard items in refresh test
Otherwise the test is not deterministic.
* Revert "clear django redis cache before each test"
This is a good thing to do, but I don't want to open up a discussion on
this PR. It was not fix for the immediate intermittent
test_refresh_cache test
This reverts commit aa52f332f3.
* format
* clickhouse trends: SET allow_experimental_window_functions = 1
* fix setting of allow_experimental_window_functions
* add cache clears for requests
* Delete SmoothingFilterLogic.ts
Removed smoothing filter logic file
* Use `SMOOTHING_INTERVALS` instead of string literal
This should ensure that if we update the url param key somewhere, we don't have to remember to change it here as well.
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* move test_trends to ee
* actually delete test_trends.py
* Transitioned smoothing logic to use Logic
* WIP: Moving away from url based logic
* WIP: Refactored logic on listeners
* fix storybook
* refactor
* set redis url
* remove keastory
* Move interval filter to setFilters insight logic
* update trend snapshots
* run prettier
* Add stickiness test deps that were in test_trends
* Fix storybook
* fix stickiness tests
* update snapshot
* fix cohort test
* only display smoothing on linear
* fix tests
* Add some tests
* added box to make dropdown consistent
* Floor
* fix tests
* fix tests
* fix test
* Add feature flag
Co-authored-by: Marcus Hyett <marcus@posthog.com>
Co-authored-by: Marcus Hyett (PostHog) <85295485+marcushyett-ph@users.noreply.github.com>
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
Co-authored-by: Tim Glaser <tim@glsr.nl>
ClickHouse 21.9 introduced heredoc syntax. The funnel queries have
repeated sections, one for each step within the funnel. As a result we
can end up in a situation where we create queries that include heredocs,
and as a result are invalid SQL queries.
This test just adds a test that is designed to produce a query that
would, assuming the implementation, produce a heredoc string.
* Dont use trim() in clickhouse
trim broke in 21.12 and if we want to use if going forward, we need to
support working with a different version
* Update snapshots
* fix(clickhouse): resolve 21.8 -> 21.9 test issues
This is the minimum change to make
ee/clickhouse/queries/funnels/test/test_funnel_unordered.py::TestFunnelUnorderedStepsBreakdown::test_funnel_aggregate_by_groups_breakdown_group
pass. There may be other tests still failing.
The issue was introduced with the [addition of
heredocs](https://github.com/ClickHouse/ClickHouse/pull/26671) into the
clickhouse syntax. heredocs use `$` as delimiters. We're using `$` for
column identifiers in quite a few places, so now we need to ensure that
we have quoted these before rendering into SQL.
* update other snapshots
* update snapshots
* fix(clickhouse): quote materialized column identifiers
