* Resolve weird SQL formatting issue
* Use a discriminated union for ActorType
* Use standard response types for related groups
* Update typing
* Always filter related actors by group type index
* Update snapshots & typing
* Migration to add name_singular and name_plural to groups
* Allow modifying group type names
* Use name_plural and name_singular through the app
* Update tab names
* fixup
* Update snapshots
* add trends option to experiment
(cherry picked from commit be9b8911f9)
* connect trends experiments
* clean up some stuff
* fix test
* results chart display for trends
* fix some issues with trends
* add in loading states and different insight types results wording
* move experiment results into loader
* add progress bar and clean up loading state
* fix experiment insight type bug
* add text beneath percent bar
* typing..
* single filter hide duplicate row button
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* Don't delete events from Postgres since we don't do it on CH
* Update frontend for accuracy
* Update test_person.py
* Don't display "ID-less user" as a link
* Set time bounds for "all of time" filter
We won't display data points from before 2015 anymore, avoiding
confusion like in https://github.com/PostHog/posthog/issues/7626
* Disable dates before 2015, add tooltip
* Remove flipsort related logic
* Only load events from the last 6 months in events table
* Disable customization in actions tab
* Rename prop
* Dont poll events in actions page
* Show a different header above events table in actions page
* Update tests with frozen time
* Decouple months of actions tab with other months
* Calculate action count over 3 months
* Update tests
* Remove `minute` interval support from frontend
* Remove minute support from backend, default to hour instead
Also adds tests for interval logic
* Remove now-dead minute support from backend
* revert interval_candidate logic
* Move IntervalMixin to separate file
* Run queries against person_distinct_id2 when async migration is done
* Only write to clickhouse_person_unique_id topic if async migration is incomplete
* Update query snapshots
* Update plugin-server
* Adjust caching logic
* chore(pdi): add data migration for pdi to pdi2
This adds an async migration to copy the latest non-deleted
`(team_id, person_id, distinct_id)` tuples from `pdi` to `pdi2`.
Note that this has already be performed for tead_id = 2 on posthog,
cloud so we ensure we're maintaining parity with however this
migration was performed. I've done this by running:
```
SELECT * FROM <old_query>
FULL JOIN <new_query> new
ON old.distinct_id = new.distinct_id
WHERE old.person_id <> new.person_id
```
specifically for team_id = 2.
* Rename migration
* Skip 0003_fill_person_distinct_id2 on fresh installs
* Clarify version requirements
* Run async migrations using a while-loop instead of tail recursion
Python has a stack limit of 1000, which we might easily run into for
0003 migration
* Use built-in progress tracking
* Make description fit into database 400 char limit
* Add correctness test for new async migration
* Migrate person_distinct_id2 team-by-team
* Remove dead code
* Update migration notes
* Fix foss tests
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
* Keep person_distinct_id2 in sync on split_person
Follow-up to https://github.com/PostHog/posthog/pull/7671, covering a
new corner case.
* Deprecate some methods
* Handle versioning in split_person
* Select for update
* query events by date for properties known to be dates
* can't currently import journeys_for helper outside of ee folder
* actually remove the EE import
* even though no values of the specified property were not parseable account for unparseable values
* move tests into EE folder cos they're clickhouse only
* remove new line to take this file out of the PR diff
* use more forgiving ClickHouse datetime function
* wip
* add more tests and rename filters
* Adds a broken date picker behind a feature flag
* Revert "wip"
This reverts commit 6ec1da8744.
* add working basic UI for date selection in taxonomic property filter
* add a failing test by writing a unix timestamp the way the SDK does
* obey mypy
* add awareness of type and format so we can query our $time property as a DateTime in the events api
* hard code property definition in the eventsTableLogic
* Revert "obey mypy"
This reverts commit f09ae4369a.
* hold taxonomic property filter open - but with possible kludge in cohortsModel
* remove my edits to settings
* put back materialization of columns that test snapshots are expecting
* let the datepicker hold the taxonomic pop up open
* update snapshot
* simpler datepicker visibility in filter row
* simplify property type format for unix timestamps
* remove the datepicker now button
* Actually delete all of the drop down held open tests
* Remove leftover `console.log`s
Co-authored-by: Michael Matloka <dev@twixes.com>
* add property type and format and set them for
* add a failing test
* with passing tests on the ee property definition model
* Add a migration to set as a DateTime
* Clarify in failing test that it's only list lookup failing
* add empty property definition fields to assertions
* fix merge error
* clarify tests
* add more supported types
* create event property model
* add null
* rename cache vars
* update event properties table on ingestion
* match date formats
* match date formats
* better string handling
* property type can be null too
* pass event timestamp
* update property type later
* perform all updates through a buffer object
* move to EventPropertyCounter
* fix migration
* improve flush last seen at job
* flush job periodically + env
* upsert all event properties in 1 query
* log to statsd
* enable property counter only if experimental mode enabled
* use now() instead of event timestamp
* fix seconds
* add user/pass for default postgres
* add tests
* use big integers
* make query work with 50k props
* processing events saves event properties
* fix script
* test date format detection
* default enabled
* only enable event property counter for specific teams
* eslint fixes
* fix logs double-sync noise in tests
* fix bigint test
* don't do tasks that make no sense
* remove dead code
* proof of concept
* remove old <PropertyFilter> component
* change import paths
* event properties sorted by event names
* get event names from actions if used
* scope event property filters by event names
* fix eslint
* simpler test setup
* different contraint name
* refactor team manager
* greatly simplify the system
* fetch cached event properties
* fix team manager and timestamps
* add cached entry
* also don't cache event properties for teams that have it disabled
* remove indexes that are not going to be used
* remove unused imports
* blacked
* remember event properties with a LRU cache
* fix eslint
* clean up the last bits
* remove fields we no longer have
* add some instrumentation to figure out what services we can connect to (helps debug errors locally with kafka)
* Update frontend/src/types.ts
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* merge duplicate code
* use the right prop for event names
* mute unseen events, add info icon
* use the event-property tracker also for non-EE clients
* simplify duplicates
* add test for event properties
* add test for non-EE event properties
* add flag
* revert the label and unmute the text
* change the flag to UNSEEN_EVENT_PROPERTIES
* sort by query_usage_30_day again
* Update frontend/src/types.ts
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* use None if no event_names
* fix type
* improve pagination test
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* chore(lifecycle): add comments and CTEs to clickhouse sql
It was really big and confusing, but hopefully this clarifies a little
what is going on. As a followup PR I'll be doing some work to make the
query faster :fingerscrossed: but I think worth at least getting this
in, assuming I haven't broken any tests!
* update snapshots
* remove day references
* update snapshots
* fix(retention): add regression test for not_icontains filter
Previously if using the not_icontains filter, we were trying to double
substitute a query when fetching persons for retention. This adds a
test (and in the next commit a fix) such that we instead apply the
clickhouse substitute only once to each part of the query.
Resolves https://github.com/PostHog/posthog/issues/7747
* avoid double substitution
* Rework solution to not change ActorBaseQuery interface
* Remove unued import
* fix typing
* it works...
* clean up server side
* move logic from query to helpers/api
* typing fixes
* add tests to helpers
* test for metadata queries
* api tests
* util tests
* improve some types and move logic to utils
* Get PlayerMeta working
* tests for sessionRecordingPlayerLogic
* fix some more tests
* analytics fix
* name flip
* player util tests
* Add comments + remove logs
* Clean up some comments
* make is_active a bit cleaner
* remove some log statements
* add comment to metadata function
* some cleanup
* typescript fix
* timestamp plays follows scrubbing
* fix bug with hitting 60 seconds
* fix end position bug
* fix bugs with skip ahead
* fix some bugs
* fix white screen on buffer end
* fix buffer scrubbing bugs
* fix skip bug
* fix circular import
* rename start_window_id to last_active_window_id
* fix tests
* bug fix for no window_id case
* fix bug when scrubbing to the end of the player
* rename setBuffer + setScrub to startBuffer + startScrub
* fix(retention): fix breakdown people urls
This change returns people_url for each breakdown cohort in the
response. We also merge the initial and returning queries together,
as this makes it easier to align the people query also.
Note that I'm talking about person_id as opposed to actor_type etc.
but perhaps that can be a followup.
* clean up clickhouse params
* tidy up a little
* remove import
* remove non-breakdown specific code
* make cohort by initial event date a special breakdown case
* keep date for backwards compat
* Remove unused sql
* make test stable
* wip
* Get most of the tests working
* test(retention): remove graph retention test
We no longer need this, we have all the information we need from the
table response for retention, and can construct this on the frontend.
* revert any changes to posthog/queries/retention.py
* revert any changes to ee/clickhouse/models/person.py
* Revert posthog/queries/retention.py to merge-base
* Ensure actor id is a str
* Add type for actor serialiser for type narrowing
* run black
* sort imports
* Remove retention_actors.py
* fix typings
* format
* reverse str type
* sort imports
* rename
* split out functions
* remove deuplicate logic
* working
* fix type
* don't stringify
* fix test
* ordering doesn't matter
* trigger ci
Co-authored-by: eric <eeoneric@gmail.com>
* Make person hooks only happen in tests
* Remove a data script
* Update split_person code to handle hookless life
* Delete clickhouse data async in the background
* Make settings dynamic for MATERIALIZED_COLUMNS_ENABLED
* migrate MATERIALIZED_COLUMNS_ENABLED to this and test
* allow tests that use materialized columns to hit django model for settings
* nicer installed apps config
* update snapshot from test
* make test_property more deterministic
Co-authored-by: James Greenhill <fuzionech@gmail.com>
Co-authored-by: yakkomajuri <yakko.majuri@gmail.com>
