mirror of
https://github.com/PostHog/posthog.git
synced 2024-12-01 04:12:23 +01:00
7c4b7eb811
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Eric Duong
|
4ece4ce3e8
|
Major filter refactor (#2736)
* split params into mixins * add typing * split func * dedicated paths filter * make basefilter * restructure directories * move prop mixin * implement cached_property and fix circular import * remove unused imports * stickiness filter converted * temp patch for breakdown * correct naming for derived mixins in paths * fix types * fix types * fix casing * fix breakdown arg * merge master * add to_dict * DRY to_dict * scoped to_dict ability * fix types * add missing fields * refactor session filter * remove unused imports * remove unused imports * add stickiness filter test * fix dict for pathfilter * properly load strs * add default * standardize loading * retention filter separate * add update filter to retention * change back decorator * don't allow setting on filter date_from date_to * remove seconds on default timestamp * remove derived values * fix date formatting * fix typo |
||
Michael Matloka
|
96e4ee8512
|
Nest endpoints of project-based models under /api/project/ – LITE (#2485)
* Nest endpoints under /project/ with StructuredViewSetMixin * Rewrite URLs * isort * Update utils.py * Fix errors * Fix almoast all the errors Last left to do: shared dashboards and permission classes. * isort * Adjust for master * Add compatbility with shared dashboards * Debug ClickHouse * Remove some # type: ignores * Simplify CursorPagination * Move test base from posthog.api.test to posthog.test * Improve API structure * Bring back legacy endpoints * Fix legacy compatibility * Fix bugs and typing * isort * Fix hooks test * Try fixing errors * Fix oversight * isort * Fix problems * isort * Be more tolerant * Fix naming and remove redundant code * Fix imports * Update deleteWithUndo * Roll back * Roll back more * Update .gitignore * Rollll back * Rollllllll * back * Betterify * Address feedback |
||
|
Michael Matloka
|
d24783ece6
|
Org/projects UX enhancements (#2145)
* Improve member removal UX * Add Delete Project * Constrain project deletion in the frontend * Improve no org * Make minor fixes * Bring back changes * Add member level changing * Update test_organization_members.py * Fix org leaving edge case * Polish Project Settings UI * Allow full project deletion * Fix dynamic page title edge case * Mymymypy * Remove console.log * Remove obsolete test * Fix API constraints * Fix invite creation validation case * Add cloud/self-hosted specificity to premium feature guarding * Improve user setting logic in case of error * Use Teammates instead of Team Members * Improve org/project creation UX * Add organization ownership * Add migration * Fix redirecting to org creation * Improve clipboard toast consistency * isort * Update tests * isort * Update test_organization.py * Fix typing (somewhat) * Explain passing on ownership * Address feedback * Reverse guardPremiumFeature return * Fix typing * Fix test * Fix last typing * Fix merge conflict resolution * Remove unused code |
||
|
Eric Duong
|
fd4d66738c
|
remove feature flags (#2212) | ||
|
Michael Matloka
|
533c7df105
|
Organization<>Team<>User structure (closes #1540) (#1562)
* Backend changes to implement #1461
* Added the missing migration files
* Fixes Typecheck errors
* Refactor request.user.team_set.get() to use request.user.team
* Updated user patch method to change current_team on team id instead of name
* Merged migration file
* Changes team property to return first item in queryset
* Fixes failing tests
* Changed User api to return the ids of the teams they are currently part of
* Frontend for changing teams
* Update and simplify migrations
* Improve team and user buttons
* Make team changing backend more logical
* Improve current_team mechanics
* Update test_team.py
* Fix Team.objects.create_with_data
* Update migration
* Update tests
* Make setup_review more convenient
* Add Organization and OrganizationMembership
* Replace is_admin with level
* Extend API
* Update team.py
* Improve modeling
* Improve handling of new mechanics
* Add proper migration
* Remove _ensure_organization_and_team
* Update 0084_org_team_user.py
* Improve user, org and team creation
* Make MembershipLevel more flexible for the future
* Add member deletion
* Fix naive datetime warnings
* Update setup_review.py
* Update API route
* Make PersonalAPIKey changes
* Update models and migrations, fix typing
* Fix typing
* Use MAC-less UUID v1 instead of v4 for better performance
* Add abstract UUIDModel
* Update utils.py
* Update utils.py
* Fix multi/unicast bit
* Update APIs, frontend and tests
* Update pull_request_template.md
* Fix comment
* Fix migration error
* Compress migrations
* Updates with minimal renaming
* More updates
* Make further updates
* Update test_team_user.py
* Fix issues
* Add migration
* Satisfy mypy
* Remove Signup redirect on logged in
* Use uuid1_macless in Person
* Fix typing
* Update tests
* Update /api/team/signup to /api/organization/signup
* Fix mypy issues and update tests
The remaining failures are actually missing functionality (TDD applied), so filling these in.
* Update 0086_org_live
* Make small improvements
* Implement permissions
* Remove now unneccesary membership check
* Update setup_dev.py
* Make small frontend improvements
* Add drf-nested-routers as requirement
* Remove unused import
* Implemented nested routes
* Remove cruft
* Add relevant org/proj/user name to headings
* Fix imports
* Update migration
* Replace unreliable drf-nested-routers with drf-extensions
* Improve unset team handling
* Make org and team creation proper
* Update migration
* Fix migration
* Update TopContent
* Update command palette for new sidebar structure
* Remove deprecated demo data deletion
* Assume that each org has a project and fix typing
* Require paid plan for multiple orgs and projects
* Make HogFlix demo a separate team
* Update migration
* Slightly improve style
* Adjust page layout bottom padding
* Make user dropdown nicer
* Fix base app tests
* Satisfy mypy
* Fix test_leave_organization
* Improve wording
* Possibly fix import
* Remove misplaced None check
* Enhance org and teams APIs and add tests
* Fix /api/projects for particular Team
* Improve invites and demo data
* Address feedback
* Put everything related to billing on Organization
* Fix minor issues
* Simplify invitation creation
* Update team model
* Make orgs and projects premium only on self-hosted
* Improve testing
* Update migration
* Remove extra License import
* Fix minor issues
* Fix Django tests
* Fix Cypress
* Fix yarn build
* Fix TeamSignupViewset
* Fix posthog-production incompatibility
* Remove extraneous insight endpoint registration
* Adjust tests for posthog-production
* Simplify invitations and fix email validation
* Address all feedback
* Satisfy mypy
* Update migration
* Fix constraint removal in migration
* Update tests
* Fix test creation edge case
* Run posthog-production CI tests against this branch and teams-live
* Ensure that js_posthog_api_key is always passed
* Fix preflight check pre-login
* Update cypress tests
* Update instanceStatus.js
* Bring ee tets up to par
* Bring actions-ux-201012 back
* Cypress retry in cypress.json
* Revert "Run posthog-production CI tests against this branch and teams-live"
This reverts commit
|
||
Tim Glaser
|
80d20e385b
|
Clickhouse use elements chain (#1849)
* Clickhouse use elements chain * Fix stuff * Add action tests and start regex * Progress * Progress part deux * Fix everything * Add tag name filtering * Fix funnels * Fix tag name regex * Fix ordering * Fix type issues * Fix empty nth-child * Remove commented code * Split with semicolon and escaped quotes * Specify all select columns |
||
James Greenhill
|
379518e285
|
"Clickhouse Features V2 (#1565)" (#1750)
* initial
* migration command
* migrations working
* add modelless views for clickhouse
* initial testing structure
* use test factory
* scaffold for all tests
* add insight and person api
* add basic readme
* add client
* change how migrations are run
* add base tables
* ingesting events
* restore delay
* remove print
* updated testing flow
* changed sessions tests
* update tests
* reorganized sql
* parametrize strings
* element list query
* change to seralizer
* add values endpoint
* retrieve with filter
* pruned code to prepare for staged merge
* working ingestion again
* tests for ee
* undo unneeded tests right now
* fix linting
* more typing errors
* fix tests
* add clickhouse image to workflow
* move to right job
* remove django_clickhouse
* return database url
* run super
* remove keepdb
* reordered calls
* fix type
* fractional seconds
* fix type error
* add checks
* remove retention sql
* fix tests
* add property storage and tests
* merge master
* fix tests
* fix tests
* .
* remove keepdb
* format python files
* update CI env vars
* Override defaults and insecure tests
* Update how ClickHouse database gets evaluated
* remove bootstrapping clickhouse database routine
* Don't initialize the clickhouse connection unless we say it's primary
* .
* fixed id generation
* remove dump
* black settings
* empty client
* add param
* move docker-compose for ch to ee dir
* Add _public_ key to repo for verifying self signed cert on server
* update ee compose file for ee dir
* fix a few issues with tls in migrations
* update migrations to be flexible about storage profile and engine
* black settings
* add elements prop tables
* add elements prop tables
* working filter
* refactored
* better url handling
* add mapping table
* add processing to worker task
* working cohort with actions
* add cohort property filtering
* add cohort property filtering
* reformat and add cohort processing
* prop clauses
* add util
* add more util
* add clickhouse modifier
* Clickhouse Sessions (#1623)
* sessions sql
* skeleton
* add endpoint
* better tests
* sessions list
* merge clickhouse-actions
* added session endpoint
* sessions sql working again
* add clickhouse modifier
* session avg with props working
* add dist
* tests working (no list)
* list working
* add formatting
* more formatting
* fix tests
* dummy commit
* fix types
* remove unnecessary improt
* ignore type when importing from ee in task
* fix test running
* Clickhouse Trends Base (#1609)
* initial working
* date param almost working
* fix date range and labels
* fixed monthly math
* handle compare
* change table
* using new event ingestion
* direct query actions working
* remove interface
* fix date range
* properties initial working
* handle operator
* handle operator
* move timestamp parse
* move more to util
* inital breaking down working
* working cohort breakdown
* some tests running
* fix sessions
* cohort tests
* action and interval test
* reorder cohort filtering
* rename retention test
* fix inits
* change multitenancy tests
* fix types
* fix optional types
* replace ch_client.execute with sync_execute
* replace ch_client.execute with sync_execute, part 2
* Clickhouse Stickiness + Process Event (#1654)
* generate clickhouse uuid script
* set CLICKHOUSE_SECURE=False by default if running in TEST or DEBUG
* convert person_id to UUID, make adding `person_id` optional, add distinct_ids already in the `create_person` function
* Fix test_process_event_ee.py, remove all calls to Person.objects.*
* add back util
* fix broken imports
* improve process_event test clickhouse queries
* Basic stickiness query
* Clickhouse Stickiness tests
* stickiness test [WIP, actions fail]
* generate clickhouse uuid script
* change default test runner if PRIMARY_DB=clickhouse
* fix stickiness test for actions
* fix merge bug
* remove _create_person stub; cohort person_id is UUID now
* fix typing
* Clickhouse trends process math (#1660)
* most of process math works
* all process math
* fix ordering issue
* unusued imports
* update property comparison for process_event_ee
* indentation wrong missing calls
* demo users and events (#1661)
* finish breakdown filtering tests and reformat label function
* add increment to demo_data
* update demo data populating
* Add people endpoint for ch (#1670)
* add people endpoint for ch
* stickiness people
* fix value padding
* add process math to breakdown and
* add limit
* fix tests
* condensed code
* converted test to factory
* add people tests
* add month handling
* add typing fix
* change people test handling
* fix tests
* Clickhouse funnels 2 (#1668)
* add elements to create_event
* WIP closes #1663 Add funnels to clickhouse
* Make funnels work
* Clean up
* Move filtering around
* Add mypy tests and fix
* Performance improvements
* fix person tests again
* add people for funnel endpoint
* fix prop numbering
Co-authored-by: Marius Andra <marius.andra@gmail.com>
Co-authored-by: Eric <eeoneric@gmail.com>
* merge master
* add retention
* update types
* more typing errors
* fix types
* bug with kafka payload, elements insert, and demo data
* Clickhouse Paths (#1657)
* paths clickhouse test (fails)
* add elements to create_event
* make this fail for clickhouse
* hardcoded query that returns good results for $pageviews, no filters yet
* clean up queries
* bound by time, fix 30min new session boundary
* support screen and custom events
* add properties filter
* paths url
* filter by path start
* better path start test
* even better path start test
* start from the first "path start" in a group
* test for person_id in paths
* partition by person_id for POSTGRES paths
* partition by person_id for Clickhouse paths
* clean up order in paths test
* clean up order in paths test
* join elements
* force element order on element group creation
* remove "order" when creating elements in tests and demo
* get list of elements for paths
* add limit to paths query
* use materialized view
* rename "element_hash" to "elements_hash" (no change in db)
* cull rows that are definitely unused
* simplify query
* New highly optimized paths clickhouse query
* start_point for $autocapture paths
* extract event property values from clickhouse
* prevent crash
* select one element sql
* get elements for event
* remove lodash
* remove host from $pageview path elements if same domain as incoming path
* show metadata based on loaded paths filter, not in flight filter
* fix order (all soures and targets in order, not all sources first, then all targets after) - makes for a better looking graph
* add test that makes the Postgres paths query fail
* fix postgres paths --> no fuzzy matching, breaks "starts with" for urls and gives too many incorrect start points
* create automatic /demo urls that match the real urls (no ending /)
* fix elements queries
* path element joins
* create persons via postgres in paths test
* change serializers back to id
* fix tests with uuid
* fix demo
* more bugs
* fix type
* change now to timezone aware
* [clickhouse] retention filters (#1725)
* implemented target entity and prop filtering
* add insight view override
* fix endpoint and filters
* include tests
* fix tests
* add period filtering
* .
* fix pg param name
* add filtering params to both queries in retention sql
* fix param again
* change to todatetime
* change tz to timezone
* add back timezone in model/event
* [clickhouse] feature flag endpoint requests (#1731)
* add feature flags to endpoints
* add flags to endpoints that check on request
* remove magic strings and fill in missing flags
* fix types
* add missing flag
* change from iso
* fix more timestamps and comparator
* change _people to get_people in actions view
* remove action and cohort populating
* change inheritance
* "Clickhouse Features V2 (#1565)"
This reverts commit
|
||
|
James Greenhill
|
0b371d43ec
|
Revert "Clickhouse Features (#1565)" (#1748)
This reverts commit
|
||
|
Eric Duong
|
24713b923d
|
Clickhouse Features (#1565)
* initial * migration command * migrations working * add modelless views for clickhouse * initial testing structure * use test factory * scaffold for all tests * add insight and person api * add basic readme * add client * change how migrations are run * add base tables * ingesting events * restore delay * remove print * updated testing flow * changed sessions tests * update tests * reorganized sql * parametrize strings * element list query * change to seralizer * add values endpoint * retrieve with filter * pruned code to prepare for staged merge * working ingestion again * tests for ee * undo unneeded tests right now * fix linting * more typing errors * fix tests * add clickhouse image to workflow * move to right job * remove django_clickhouse * return database url * run super * remove keepdb * reordered calls * fix type * fractional seconds * fix type error * add checks * remove retention sql * fix tests * add property storage and tests * merge master * fix tests * fix tests * . * remove keepdb * format python files * update CI env vars * Override defaults and insecure tests * Update how ClickHouse database gets evaluated * remove bootstrapping clickhouse database routine * Don't initialize the clickhouse connection unless we say it's primary * . * fixed id generation * remove dump * black settings * empty client * add param * move docker-compose for ch to ee dir * Add _public_ key to repo for verifying self signed cert on server * update ee compose file for ee dir * fix a few issues with tls in migrations * update migrations to be flexible about storage profile and engine * black settings * add elements prop tables * add elements prop tables * working filter * refactored * better url handling * add mapping table * add processing to worker task * working cohort with actions * add cohort property filtering * add cohort property filtering * reformat and add cohort processing * prop clauses * add util * add more util * add clickhouse modifier * Clickhouse Sessions (#1623) * sessions sql * skeleton * add endpoint * better tests * sessions list * merge clickhouse-actions * added session endpoint * sessions sql working again * add clickhouse modifier * session avg with props working * add dist * tests working (no list) * list working * add formatting * more formatting * fix tests * dummy commit * fix types * remove unnecessary improt * ignore type when importing from ee in task * fix test running * Clickhouse Trends Base (#1609) * initial working * date param almost working * fix date range and labels * fixed monthly math * handle compare * change table * using new event ingestion * direct query actions working * remove interface * fix date range * properties initial working * handle operator * handle operator * move timestamp parse * move more to util * inital breaking down working * working cohort breakdown * some tests running * fix sessions * cohort tests * action and interval test * reorder cohort filtering * rename retention test * fix inits * change multitenancy tests * fix types * fix optional types * replace ch_client.execute with sync_execute * replace ch_client.execute with sync_execute, part 2 * Clickhouse Stickiness + Process Event (#1654) * generate clickhouse uuid script * set CLICKHOUSE_SECURE=False by default if running in TEST or DEBUG * convert person_id to UUID, make adding `person_id` optional, add distinct_ids already in the `create_person` function * Fix test_process_event_ee.py, remove all calls to Person.objects.* * add back util * fix broken imports * improve process_event test clickhouse queries * Basic stickiness query * Clickhouse Stickiness tests * stickiness test [WIP, actions fail] * generate clickhouse uuid script * change default test runner if PRIMARY_DB=clickhouse * fix stickiness test for actions * fix merge bug * remove _create_person stub; cohort person_id is UUID now * fix typing * Clickhouse trends process math (#1660) * most of process math works * all process math * fix ordering issue * unusued imports * update property comparison for process_event_ee * indentation wrong missing calls * demo users and events (#1661) * finish breakdown filtering tests and reformat label function * add increment to demo_data * update demo data populating * Add people endpoint for ch (#1670) * add people endpoint for ch * stickiness people * fix value padding * add process math to breakdown and * add limit * fix tests * condensed code * converted test to factory * add people tests * add month handling * add typing fix * change people test handling * fix tests * Clickhouse funnels 2 (#1668) * add elements to create_event * WIP closes #1663 Add funnels to clickhouse * Make funnels work * Clean up * Move filtering around * Add mypy tests and fix * Performance improvements * fix person tests again * add people for funnel endpoint * fix prop numbering Co-authored-by: Marius Andra <marius.andra@gmail.com> Co-authored-by: Eric <eeoneric@gmail.com> * merge master * add retention * update types * more typing errors * fix types * bug with kafka payload, elements insert, and demo data * Clickhouse Paths (#1657) * paths clickhouse test (fails) * add elements to create_event * make this fail for clickhouse * hardcoded query that returns good results for $pageviews, no filters yet * clean up queries * bound by time, fix 30min new session boundary * support screen and custom events * add properties filter * paths url * filter by path start * better path start test * even better path start test * start from the first "path start" in a group * test for person_id in paths * partition by person_id for POSTGRES paths * partition by person_id for Clickhouse paths * clean up order in paths test * clean up order in paths test * join elements * force element order on element group creation * remove "order" when creating elements in tests and demo * get list of elements for paths * add limit to paths query * use materialized view * rename "element_hash" to "elements_hash" (no change in db) * cull rows that are definitely unused * simplify query * New highly optimized paths clickhouse query * start_point for $autocapture paths * extract event property values from clickhouse * prevent crash * select one element sql * get elements for event * remove lodash * remove host from $pageview path elements if same domain as incoming path * show metadata based on loaded paths filter, not in flight filter * fix order (all soures and targets in order, not all sources first, then all targets after) - makes for a better looking graph * add test that makes the Postgres paths query fail * fix postgres paths --> no fuzzy matching, breaks "starts with" for urls and gives too many incorrect start points * create automatic /demo urls that match the real urls (no ending /) * fix elements queries * path element joins * create persons via postgres in paths test * change serializers back to id * fix tests with uuid * fix demo * more bugs * fix type * change now to timezone aware * [clickhouse] retention filters (#1725) * implemented target entity and prop filtering * add insight view override * fix endpoint and filters * include tests * fix tests * add period filtering * . * fix pg param name * add filtering params to both queries in retention sql * fix param again * change to todatetime * change tz to timezone * add back timezone in model/event * [clickhouse] feature flag endpoint requests (#1731) * add feature flags to endpoints * add flags to endpoints that check on request * remove magic strings and fill in missing flags * fix types * add missing flag * change from iso * fix more timestamps and comparator * change _people to get_people in actions view * remove action and cohort populating Co-authored-by: James Greenhill <jams@uber.com> Co-authored-by: Marius Andra <marius.andra@gmail.com> Co-authored-by: Tim Glaser <tim.glaser@hiberly.com> |