* Emit timing gauge to statsd for sync_execute on clickhouse
* take out of block for SHELL_PLUS_PRINT_SQL
* add docker testing for ease
* comment out cypress projectid
* Enable PLUGIN_SERVER_INGESTION_HANDOFF = get_bool_from_env("PLUGIN_SERVER_INGESTION_HANDOFF
* Don't set PLUGIN_SERVER_INGESTION_HANDOFF in worker
* Add comments
* Remove _HANDOFF from PLUGIN_SERVER_INGESTION
* add stats counter for plugin server handoff, so we can verify events out and events in
* add whitelisted posthog and kea organizations
* disable ingestion this round --> first let's just check the plugin server can talk to kafka & clickhouse before sending real events to it
* enable ingestion in docker-compose.ch.yml
* eliminate bad merge
* async action event matching when using postgres plugin server ingestion (#3182)
* fix org
* remove _HANDOFF from topic
* add plugin_ to plugin server ingestion topic
* update plugin server to 0.7.0
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Unify clickhouse and pg demo data creation
* Adapt yakkos demo data generation to normal demo
* Split demo.py into subfolder
* Add original, web data import
Note that elements are not added because they can't be added
automagically :(.
* Improve wording
* dont generate dashboards
* try get tests passing
* fix types
* Get cypress tests green
* Move adding feature flags to event to capture endpoint
* Remove unused imports
* Don't create empty event properties dict
* Don't ever create event properties if dict not present
* Test feature flags in test_capture
* Ensure that every event has a properties dict
* Only apply PLUGIN_SERVER_INGESTION_HANDOFF to plugin-whitelisted orgs
* Adjust tests for ensured event properies dict
* Add setting for handing off process_event_ee to plugin server
* Add StatsD settings to KEYS
* bin/plugin-server → start-plugin-server & docker-plugin-server
* Simplify to only add docker-plugin-server
* Bring back original comment
* Turn down verbosity of plugin server install
* Remove redundant if
* Fix comment
* Remove lone newline
* Roll back unsafe script changes
* Simplify dockerized plugins
* Add some depends_on
* Clarify HAND_OFF_INGESTION env var
* Use posthog-plugin-server 1.0.0-alpha.1
* Enhance bin/plugin-server and rm bin/docker-plugin-server
* Move around PLUGIN_SERVER_INGESTION_HANDOFF ifs
* Use posthog-plugin-server@1.0.0-alpha.2
* Support kafka+ssl:// in plugin-server
* Produce to topic events_ingestion_handoff for plugin server
* Use posthog-plugin-server@1.0.0-alpha.3
* Don't import Kafka topics in FOSS
* Use @posthog/plugin-server
* Update yarn.lock
* Add commands for external ClickHouse setup/teardown
* Actually delete test CH teardown command
* ClickhouseTestRunner.setup_test_environment() in setup_test_clickhouse
* Rework test setup script to work with Postgres too
* Restore master plugins dir for merge
* Unset PLUGIN_SERVER_INGESTION_HANDOFF in docker-compose.ch.yml
* Fix unimportant typo
* Build log_event data dict only once
* Make it clear in bin/plugin-server help that it's bin
* Space space
* Add model for session recording viewed
* Save view when querying for session recording data
* Send information to FE about whether session has been viewed
* Allow filtering by "recording unseen"
* Rename property
* Update migration
* Add user param for clickhouse if provided
* remove duplicate setting - forgot it was there!
* inconsistent naming between migrations and clickhouse-driver
* Highlight rows from sessions which are matched by the filter
* Send start_time of recording to frontend
This helps us calculate offsets a bit better
* Use timestamp by server for date shown
* Load session events via kea logic
* Update rrweb, rrweb-player
* Highlight events user is filtering for in sessions player
* Handle empty case properly
* Add positive test
* Order session recording events in query
* Fix filtering by multiple events with differing names
previously only the first would have been used due to overlapping
params.
* Return all highlighted times as action_filter_times
* Send back ids not timestamps
This avoids weird rounding errors causing issues
* Show skeleton for longer
* update typing
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Event must be a string
* test that event can't be a dict
* convert dicts and other unknown event names to strings
* mypy
* update packages to have mypy working with python 3.9
* fix event name sanitization types
* fix misc type error from mypy/django-stubs update
* fix other type errors from mypy/django-stubs update
* add migration to update events for users if there's weird data in team.event_names
* ignore type to fix mypy issue
* fix migrations
* Make it possible to filter by (multiple) action filters in postgres
Session will now contain "action_filter_times" key which lists when each
action filter occurred for the first time within the session.
This will be used to highlight rows/show special values in sessions
player.
* Clickhouse: support multiple action filters
* Remove dead code
* Upload static cohort using CSV, closes #2868
* fix tests
* Fix tests
* Fix e2e test
* Avoid double inserts
* Speed up query
* Move to params
* fix tests
* Use JSON instead of protobuf
* unused import
* produce instead of produce_proto
* Insert directly into clickhouse, no need for kafka
* a few quick fixes
* insert in batch to clickhouse
* test SQLi since we can't really trust what we are inserting
* Extra check for duplicates
Co-authored-by: James Greenhill <fuziontech@gmail.com>
* Remove (apparently) useless person joins from sessions sql
* WIP: Make sessions list query use python iteration
* WIP: Show loader while session data is loading
* Aggregate together sessions
* Calculate start and end points of session separately
* Remove cruft code
* Load session events asynchronously for self-hosted
Note clickhouse behavior is unchanged.
* Update pagination logic for sessions
In addition to offset, postgres now returns a dict containing person_id,
timestamp which is used to make sure we filter events on different pages
correctly
* Add some tests to SessionListBuilder
* Fix typing errors
* Fix pagination limit
* Move tests to right file
* Query less events for sessions list
Since we're ordering by end_time we know events before last end_time are
all processed.
* Add test for current_url behavior
* Make sure old tests remain working
* Remove unused base class
* Move sessions-related queries to separate subfolder
* Extract sessions list code to separate file(s)
* Sort sessions by end time in ch as well
* List end time in sessions table
* Return person email when querying sessions lists on postgres
This gets used by the view
* Return email over all user properties for sessions in clickhouse and view
* Fix an ordering bug
* Fix a pagination bug
* Fix endpoint
* Fix basic sessions tests for pagination
* Sort consistently for sessions list builder
* Roll pagination into filters
* do proper interval rounding on normal trends
* patch inconsistency
* remove microseconds
* conditionall round interval
* adjust how date_from is handled
* add retention test
* edits and split test
* Refactor adding filtering component
* Fix a type error
* Rename events to entries
This makes much more sense since you can select e.g. actions
* Start of the session filter box
* WIP: Sessions filtering
* Make SelectBox more generic
* Make sessions filtering visuals
* Extract operator value selection code
* Make action properties filterable
* Make it possible to filter by cohorts
* Filtering by events
* Make loading sessions possible with the new logic
* Make it possible to filter by actions/events via backend with new code
Note that it's all relying on old 'paths' - will refactor these later.
* Make it possible to filter by user properties
* Update imports
* Small fixes for sessions filter
* Add UserFilter file
* Ignore extra args to Property
* Make it possile to filter by recording duration
* Update how session duration filter works in the backend
* Rename a few columns
* Remove dead mixins
* Remove duplicated logic, put code behind feature flag
* Move sessions filters logic to subfolder
* Don't show properties if feature flag.
* Add saved filters support
This will require some discussion with Paolo - probably want to show
this on the side?
* Add divider before sessions table
* Narrower typing for recordings
* Some frontend for saving filters
* Small rename
* Add SessionsFilter model
* Save filter on backend
* add missing filter
* Extract const
* Make ActionFilterDropdown be able to pick actions again.
* Fix links from person modal to sessions
* Update tests.
* Fix database model
* Use proper deep equal for detecting active filter
* Remove unneeded field
* Only return filters from correct page
* Add missing data-attr
* Add instrumentation
* rename component
* Fix analytics metadata
