We'll be using qemu that ships with docker for mac. I'd updated this at
the same time as zookeeper but obviously never removed the kafka
container from before somehow.
* WIP: Create new property types for simplified cohorts
* Add documentation on simplified_cohort_filter_properties
* Handle static-cohort/precalculated-cohort property types
* Handle new property filters properly
* Add casting
* Test cohorts in more cases
* Fix a bug
* Fix benchmark simplifying
* Avoid redoing work every setup for benchmarks
* Update typing;
* Remove unneeded scope
* Add tests for simplifying and cohorts
* Roll more of "do we need to join persons table" behavior into ClickhousePersonQuery class
* Handle precalculated cohort logic in sessions
* Simplify event query
* More tests without any JSONExtract
* Simplify entity properties as well
* Improve docstring
* Add test for breakdown & precalculated cohorts
* Add test for filtering sessions by precalculated cohorts
* Reset unneeded change
* Update cohort
* Solve some typing issues
* Update benchmarking
* Fix cohort filtering tests
* Fix cohort tests
* Fix a caching issue
* Typecheck
* Handle exclusion filters
* Simplify filters code
* Simplify filters ASAP if filter is created
* Simplify route
* Remove simplification-specific logic from queries
* Remove recursion, update tests
* Pass team in more cases
* Update column optimizer specs
* Test simplify
* Update trends test
* Fix rebase fail
* events dead letter queue CH table
* format
* update schemas
* also store raw payload
* better naming
* make table name more clear
* wip better testing
* remove unused imports
* remove kafka test
* prevent non null test from running on CH migrations
* add kafka testing
* minor tests cleanup
* test naive longer sleep
* make test end-to-end
* address review
* update ttl, format
* refactor delay func, address review
I've been getting lot's of errors with the qemu emulated x86 version.
I'm just changing the arm docker-compose as I don't want my issues with
local dev to affect others.
* fix(correlation): ensure correlation partitioned by team
Previously if distinct_id's were the same between two teams, we'd end up
pulling in the event data between the teams.
* dev(tests): clear cache between partition calls
* fix(correlation): add lower bounds for selected events
Previously we would consider all events for correlation calculation. Now
we use the funnel `date_from` as the lower bounds.
* chore(correlation): exclude funnel steps
* chore(correlation): make sure cache is cleared before each test
* Update funnel success comment
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* fix: filter events by team_id
* chore(correlation): remove CTEs from correlation query
There seems to be an issue with the CTEs and production clickhouse, see
https://github.com/ClickHouse/ClickHouse/issues/29748
Instead of risking it, I'm just removing them.
* chore: update entities -> events for funnel step exclusion
* fix team_id = team_id issue
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
* chore(diagnose): add a stub endpoint for event diagnosis
This adds an insight endpoint that takes a `target_entity` and returns
a list of events ordered by significance of a person reaching
`target_entity`
Followup PRs will add actual calculations but this should act as a
conversation piece around the structure of request and response, as well
as a stub from which UI development can start.
* play around to get mypy typings right
* Sort out test reponse structurea
* refactor: address CR comments
* feat(diagnose): add diagnose stubs for implementation
* feat(funnel): add event correlation calculation implementation
This adds to the `/api/projects/<team_id>/funnel/correlation` endpoint
an implementation that calculates the odds_ratios for each event that a
user that has been part of a funnel, successful or otherwise.
* chore(correlation): get query working
* refactor(correlation): move functions to Query methods
* Always allow filtering on distinct id
* fix
* fix
* remove accidentally commited test
* include table name
* add table specific because distinct_id doesn't exist on person table
Co-authored-by: eric <eeoneric@gmail.com>
* make sure funnel and path dates are always aligned and add api handling for funnel filter persons
* use post instead of GET for paths
* parse label
* change type
* use post in test
* fix types
* test both get and post
Our queries now contain a lot of comment noise that's not relevant when
developing. This change makes PRINT_SQL env variable usable again by
stripping these comments.
* dev(clickhouse): strip out comments before executing sql
This is so we can easily copy/paste from e.g. Metabase by querying the
system.query_log. In metabase is doesn't display new lines (although you
can download to file for this), but it's not very practical.
* test(clickhouse): use `capture_select_queries` in comment strip test
* test(clickhouse): only sub. params if non-insert query
This parallels `clickhouse_driver` behaviour.
* chore(clickhouse): move sql preparation to dedicated function
* refactor: rearrange func and type definitions
* Add asv code
* Get a benchmark running
* better output folder
* Move benchmark file
* Cleanup of config
* Run a query and benchmark
* Skip benchmarks for isort
* Set up materialized columns before test
* Better skipping logic
* Add first proper benchmark, add some documentation
* Person property filtering
* Add new workflow
* Show stderr in test run
* Continue debugging actions
* Try to save benchmark results to separate repo
* Output to main directory
* Scheduling
* Remove if temporarily
* Int for ms
* Comment on the PR
* Collapsible section
* Update README
* Clarification
* Remove h3
* Remove pyproject.toml, .gitignore changes that are unneeded
* Add project-nested version of all project-dependent endpoints
* Fix hook `perform_create`
* Adjust `plugin-configs` to adhere to style of other endpoints
* Test Kafka
* black format python
* fix imports
* add kafka and zk deps for testing
* Include ZK and Kafka for all tests
* fix signature for kafka helper
* Connect to localhost for kafka
* update kafka host for all test runs
* Wrong env var for kafka
* consolidate env vars for github actions
* set the advertised hostname from the broker to localhost
* add env var to docker-compose for kafka broker advert host
* resort to what we do locally with /etc/hosts
* Remove configs for kafka that won't be used
* restore progress
* working connection no UI visual
* working for events
* edit endpoint to start point
* working with close button
* funnel to path work (no dropoff yet)
* run prettier
* make suggested adjustmenst
* Refactor `AvailableFeature` from strings to an enum everywhere
* Fix circular dependency and type
* Add "Per-project access" feature flag, premium feature, and organization switch
* Rename `OrganizationMembershipLevel` to `OrganizationAccessLevel`
* Create `ExplicitTeamMembership` model
* Show whether projects are restricted in the project switcher
* Update organizations API code
* Fix migrations
* Move organization tests that require EE to `ee`
* Revert `OrganizationMembershipLevel` rename
* Fix organization tests
* Update migration
* Fix schema and add Members to Project Settings
* Build out test memberships API with security tests
* Update `TeamMembers` and `teamMembersLogic`
* Move "Per-project access" description to tooltip
* Add moar tests
* Fix Project Members list logic
* Add additional membership checks
* Update migrations
* Fix typing
* Adjust explicit team memberships API similarly
* Fix typo
* Unify `ExplicitTeamMemberSerializer`
* Remove old changes to `membersLogic` usage
* Use `effective_membership_level` on `TeamBasicSerializer`
* Clean up organization update tests
* Explicitly disallow enabling per-project access for free
* Fix circular import
* Remove `id` from `UserSerializer`
* Fix typing
* Try to fix import
* Fix fatal typing
* Add more tests
* Update permissioning.ts
* Add clarifying comment to migration
* Fix import
* Revert `TopNavigation` changes
* Restore `TopNavigation` changes
This reverts commit 05fd9e4ed6.
* Try to add a story for `TopNavigation`
* minor clarifications
* Revert `TopNavigation` changes
* Restore `TopNavigation` changes
* Make new access control entirely project-based
* Update migrations
* Add `project_based_permissioning` to `TeamBasicSerializer`
* Update test_team.py
* Fix `isRestricted` in `ProjectRow`
* Disable project creation for non-admins
* Make project icon in top nav itself dynamic as well
* Fix story
* Delete 0169_project_based_permissioning.py
* Apply suggestions from code review
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Update frontend/src/layout/navigation/TopNavigation.tsx
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Address feedback
* Project-based permissioning full data access restrictions (#6068)
* Fix Access Control restriction tooltip
* Add `TeamMemberAccessPermission` and use it in viewsets
* Add `ErrorProjectUnavailable` scene
* Ignore mypy
* Update MainNavigation.tsx
* Update explicit_team_member.py
* Fix frontend detection of unavailable project
* Fix some tests and edge cases
* Fix basic permissions
* Add more tests
* Simplify `ExplicitTeamMemberViewSet` permissions
* Improve restrictions and add moar tests
* Update frontend
* Fix a couple of things
* Fix import
* Fix some edge cases
* Fix typing errors
* Use hedgehog instead of moth
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Address feedback
* Add proper permissioning to dashboard views
* Update ee/api/test/test_dashboard.py
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Run prettier
* Remove debug code
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Move Sessions tab -> Session Recording (behind feature flag)
Note: the /scenes/sessionRecordings folder is a copy of /scenes/sessions. Planning to remove sessions from the new SessionRecordings folder over the next coupld of commits
* Add very basic session recording page
* Add session recordings to persons page and session recs can play
* add endpoints for clickhouse
* fix ordering in existing tests
* add basic test to pg and clickhouse
* small bug fix
* type fix
* add test for data leaking between teams
* fix navigation bugs
* Fix name of logic
* Add basic logic tests
* fix tests
* fix laoding state bug
* revert changes to existing session queries
* move to proper queries for session recordings
* type fixes
* rename feature flag
* type fix
* Fix capitalization
* remove showSessions and clean up ifs
* Style changes and refactor SessionPlayerDrawer
* bug fix
* move to using query params
* remove type from persons logic
* Move sessionRecordingTableLogic key to distinctId
* rename setSessionRecordingId to openSessionPlayer
* improve tests
* Cleanup from review
* move team and limit to parameters
* new path tab querybuilder
* prettier
* clean up
* wip
* format querybuidler
* prepare path type filter component
* dropdown
* formatted popup
* change button for endpoints
* styling the type buttons
* logic for all left side querybuilder components
* exclusion UI and logic
* improve exclusion handling, ensure API calls are made
* API fixes for event types
* fix popup on close issue with exclusions
* hide overflow on buttons
* all ui working
* more formatting adjustments
* type fixes
* leave response ambiguous
* feature flagging and adjust styles
* Add Step limit option
* add tooltip to wildcards
* rewrite tooltip for exclusions
* make Events toggle responsive
* fix types
* responsive at xl instead of lg for no overlaps
* Proof of concept of connecting Funnel to Paths
* protecc wip
* Connect Funnels to Paths
* remove funnel button for now
* remove unused
Co-authored-by: eric <eeoneric@gmail.com>
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
