Fixes https://github.com/PostHog/posthog/issues/3282
The problem was in pagination as expected.
Ran the query against our team in production clickhouse - it now returns
expected data and speed did not seem to change much.
* Emit timing gauge to statsd for sync_execute on clickhouse
* take out of block for SHELL_PLUS_PRINT_SQL
* add docker testing for ease
* comment out cypress projectid
* Enable PLUGIN_SERVER_INGESTION_HANDOFF = get_bool_from_env("PLUGIN_SERVER_INGESTION_HANDOFF
* Don't set PLUGIN_SERVER_INGESTION_HANDOFF in worker
* Add comments
* Remove _HANDOFF from PLUGIN_SERVER_INGESTION
* add stats counter for plugin server handoff, so we can verify events out and events in
* add whitelisted posthog and kea organizations
* disable ingestion this round --> first let's just check the plugin server can talk to kafka & clickhouse before sending real events to it
* enable ingestion in docker-compose.ch.yml
* eliminate bad merge
* async action event matching when using postgres plugin server ingestion (#3182)
* fix org
* remove _HANDOFF from topic
* add plugin_ to plugin server ingestion topic
* update plugin server to 0.7.0
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Unify clickhouse and pg demo data creation
* Adapt yakkos demo data generation to normal demo
* Split demo.py into subfolder
* Add original, web data import
Note that elements are not added because they can't be added
automagically :(.
* Improve wording
* dont generate dashboards
* try get tests passing
* fix types
* Get cypress tests green
* Move adding feature flags to event to capture endpoint
* Remove unused imports
* Don't create empty event properties dict
* Don't ever create event properties if dict not present
* Test feature flags in test_capture
* Ensure that every event has a properties dict
* Only apply PLUGIN_SERVER_INGESTION_HANDOFF to plugin-whitelisted orgs
* Adjust tests for ensured event properies dict
* Add setting for handing off process_event_ee to plugin server
* Add StatsD settings to KEYS
* bin/plugin-server → start-plugin-server & docker-plugin-server
* Simplify to only add docker-plugin-server
* Bring back original comment
* Turn down verbosity of plugin server install
* Remove redundant if
* Fix comment
* Remove lone newline
* Roll back unsafe script changes
* Simplify dockerized plugins
* Add some depends_on
* Clarify HAND_OFF_INGESTION env var
* Use posthog-plugin-server 1.0.0-alpha.1
* Enhance bin/plugin-server and rm bin/docker-plugin-server
* Move around PLUGIN_SERVER_INGESTION_HANDOFF ifs
* Use posthog-plugin-server@1.0.0-alpha.2
* Support kafka+ssl:// in plugin-server
* Produce to topic events_ingestion_handoff for plugin server
* Use posthog-plugin-server@1.0.0-alpha.3
* Don't import Kafka topics in FOSS
* Use @posthog/plugin-server
* Update yarn.lock
* Add commands for external ClickHouse setup/teardown
* Actually delete test CH teardown command
* ClickhouseTestRunner.setup_test_environment() in setup_test_clickhouse
* Rework test setup script to work with Postgres too
* Restore master plugins dir for merge
* Unset PLUGIN_SERVER_INGESTION_HANDOFF in docker-compose.ch.yml
* Fix unimportant typo
* Build log_event data dict only once
* Make it clear in bin/plugin-server help that it's bin
* Space space
* Add model for session recording viewed
* Save view when querying for session recording data
* Send information to FE about whether session has been viewed
* Allow filtering by "recording unseen"
* Rename property
* Update migration
* Add user param for clickhouse if provided
* remove duplicate setting - forgot it was there!
* inconsistent naming between migrations and clickhouse-driver
* Highlight rows from sessions which are matched by the filter
* Send start_time of recording to frontend
This helps us calculate offsets a bit better
* Use timestamp by server for date shown
* Load session events via kea logic
* Update rrweb, rrweb-player
* Highlight events user is filtering for in sessions player
* Handle empty case properly
* Add positive test
* Order session recording events in query
* Fix filtering by multiple events with differing names
previously only the first would have been used due to overlapping
params.
* Return all highlighted times as action_filter_times
* Send back ids not timestamps
This avoids weird rounding errors causing issues
* Show skeleton for longer
* update typing
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Event must be a string
* test that event can't be a dict
* convert dicts and other unknown event names to strings
* mypy
* update packages to have mypy working with python 3.9
* fix event name sanitization types
* fix misc type error from mypy/django-stubs update
* fix other type errors from mypy/django-stubs update
* add migration to update events for users if there's weird data in team.event_names
* ignore type to fix mypy issue
* fix migrations
* Make it possible to filter by (multiple) action filters in postgres
Session will now contain "action_filter_times" key which lists when each
action filter occurred for the first time within the session.
This will be used to highlight rows/show special values in sessions
player.
* Clickhouse: support multiple action filters
* Remove dead code
