0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-12-01 12:21:02 +01:00
Commit Graph

207 Commits

Author SHA1 Message Date
James Greenhill
2935a65dc5
MV -> View for events_with_array_props_view and remove EVENT_PROP_TABLE_SQL (#2766)
* MV -> View for events_with_array_props_view

* sort

* convert another mv into view because mv's are not triggered by views

* pedantic - no mat in naming

* remove unused tables
2020-12-15 10:59:28 -08:00
Eric Duong
826182b8d2
Fix total aggregate values (#2723)
* postgres working

* change result attr name

* adhoc solution for clickhouse query

* add breakdown filter test

* adhoc breakdown solution

* add breakdown default

* fix frontend logic

* breakdown refactoring

* reshape breakdown functions

* remove print

* remove dump.rdb

* replace magic string with constant
2020-12-15 18:37:25 +01:00
Eric Duong
5845e3ec8b
Move lifecycle tests to separate file (#2765)
* put lifecycle in separate file

* use list
2020-12-15 16:05:38 +01:00
Karl-Aksel Puulmann
fe8b5e0dd7
Optimize breaking down by event property value (#2767)
* Optimize breaking down by event property value

Just getting the top values took 10+ seconds previously for our team,
now down to 2, and the breakdown query went from 6s -> 2s.

* solve typecheck issue
2020-12-15 13:11:35 +02:00
Michael Matloka
7ba9f7de09
Plugin server ingestion base (#2732)
* Add relevant settings to KEYS in bin/plugins-server

* Log all EE events to events_handoff Kafka topic for plugin server

* Clean up settings

* Fix FOSS

* Don't introduce KAFKA_EVENTS_HANDOFF

* Add cosmetic newline

* Add DEBUG WAL print()
2020-12-14 16:05:18 +01:00
Tim Glaser
560fe0765e
Deprecate dashboard item type and move to display (#2715)
* Deprecate dashboard item type and move to display

* Mypy rerors

* fix test

* fix

* Fix test

* Fix another test

* Remove determineInsightType

* Get rid of RetentionTable display types

* Fix update dashboard
2020-12-11 20:12:52 +01:00
Tim Glaser
803efc3489
Make active feature flags more obvious (#2742)
* Make active feature flags more obvious

* Also fix ch

* Fix typing

* Update usage URL

* Fix CH
2020-12-11 12:27:45 -03:00
Karl-Aksel Puulmann
3f0f051758
Sessions filtering postgres support (#2728)
* Duplicate existing endpoints under /api/event/sessions

* Use different endpoint for sessions table logic in sessions page

* drop irrelevant code for insights endpoint

* update documentation

* extract method

* move code under only relevant path

* Extract clickhouse sessions logic

* Refactor: Separate session list logic from rest of sessions

* move sessions list to separate file

* Move sessions list tests to separate file

* add clickhouse session list tests

* add support for duration filtering on postgres

* move api tests to right place

* kill dead code
2020-12-11 11:01:14 +02:00
Karl-Aksel Puulmann
d73e808be4
Sessions filtering: improve UX (#2726)
* Handle empty duration input properly

* rename no option

* Add close button to the filter

* Allow choosing time unit in dropdown

* solve mypy errors

* minor changes

* show arrow

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-12-11 10:44:47 +02:00
Eric Duong
952f6e5870
Add clickhouse query for people breakdown (#2669)
* add clickhouse query for people breakdown

* remove log

* add pg implementation

* remove caching and fix timestamp formatting

* modify how people are returned

* reimplement frontend logic for handling new people payload

* remove period check

* change params

* add returndict

* DRY retention param formatting

* remove feature flag for retention

* remove feature flag for retention
2020-12-10 14:51:24 +01:00
Karl-Aksel Puulmann
b03d4da7e0
Allow filtering sessions by recording duration (#2721)
* Use return value of add_session_recording_ids

The mutation here is incidental, this is symmetrical with how postgres
behaves.

* Implement sessions_filter

This will hold session-recording specific filters

* WIP: make filtering by session recording length possible on ch

* Kill dead method

* Allow filtering by session length

* Make logic a bit smoother around filtering by recording duration

* More common code for clickhouse_session_recording

* Add filtering tests for ch

* Resolve mypy issues

* Only run duration tests for ch

* Put new filters logic behind a feature flag

* Kill dead const

* Solve operators-related comment

* Review feedback: rename variable

* Fix test failures
2020-12-10 10:26:47 +02:00
Eric Duong
d3735cb15e
add all time support for lifecycle (#2693) 2020-12-09 19:31:48 +01:00
Eric Duong
25d60bf839
Fix prop filtering when action has props (#2711)
* add prepend clause to prop filtering

* add defaults and parametrize an event field

* add test and make the fix generalized
2020-12-09 13:19:58 -05:00
Michael Matloka
1c80aca907
Cleaned up system status page (#2682)
* Polish system status page

* Show system health badge to is_staff users on Cloud

* Update Cypress test

* Correct inconsistent scene name

* Use RDBMS enum instead of constants

* Add "Analytics database in use" to System Status

* Fix typing
2020-12-09 13:57:12 +01:00
Michael Matloka
b39dce42cf
Fix webpack-dev-server inside Docker (#2699) 2020-12-08 11:44:39 +01:00
Michael Matloka
46d21b8eeb
Zapier updates (#2686)
* Update schema used in action_performed REST hook

* Optimize find_and_fire_hook

* Isort

* Fix Event.objects.filter_by_event_with_people in process_entity_for_events

* Revert typing changes
2020-12-07 16:29:58 +01:00
Tim Glaser
f22a082f30
Debug CH queries (#2666)
* Debug CH queries

* tests

* Logout when impersonated session

* Put "Debug ClickHouse queries" in its own command

* Clean up ClickHouse modal

Co-authored-by: Michael Matloka <dev@twixes.com>
2020-12-07 16:06:14 +01:00
Eric Duong
8fbbe679f5
Stickiness improvement and filter refactor (#2638)
* stickiness filter refactor

* stickiness clickhouse

* parametrize clickhouse trunc

* add interval tests

* fix type and casing

* change name to interval not period

* change defaults

* remove offsets

* move stickinses people endpoint

* move imports

* remove unused imports

* fix time defaults

* swap endpoint

* add interval tests

* move api test

* fix all time calculation and add team_id filter to earliest timestamp ch
2020-12-04 20:42:01 +01:00
Michael Matloka
86c0c1af91
Make Google login an Enterprise/Cloud feature (#2501)
* Make Google login an Enterprise OR Cloud feature

* Fix test

* isort

* Fix typing

* Restore package.json
2020-12-04 19:54:27 +01:00
Tim Glaser
adcf493202
Fix ee webhook (#2664)
* Fix ee webhook

* Get rid of warnings

* Add webhooks
2020-12-04 16:43:22 +01:00
Tim Glaser
d60799c4f8
Fix nan values (#2665)
* Fix nan values

* unused imports
2020-12-04 16:27:08 +01:00
Eric Duong
b9d6112058
add date squeezing (#2657) 2020-12-04 11:02:52 +01:00
James Greenhill
ed6eb5e796
Setup ecs configs for web, worker, migration tasks and services (#2458)
* add worker to the ecs config and deploy

* for testing

* pull from this branch for testing

* chain config renders

* split out events pipe

* Set is_heroku true because of heroku kafka

* update /e/ service to run on port 8001

* add 8001 to the container definition as well

* simplify

* test migrating w/ ecs task using aws cli

* split services

* typo in task def

* remove networkConfiguration from task definition

* duplicate

* task-def-web specific

* update events service name

* Handle base64 encoded kafka certs

* if it's empty then try to set it for env vars

* fix b64 decode call

* cleanups

* enable base64 encoding of keys for kafka

* depend on kafka-helper for deps

* reformat

* sort imports

* type fixes

* it's late, I can't type. typos.

* use get_bool_from_env

* remove debug bits. Trigger on master/main

* prettier my yaml

* add notes about ref in GA

* up cpu and memory
2020-12-03 15:51:37 -08:00
Eric Duong
0feac747cd
Lifecycle prop filtering (#2634)
* add filters to ch

* add tests and implement for postgres

* fix postgres implementation
2020-12-02 14:15:51 -05:00
Eric Duong
997ec36916
Lifecycle Graph (#2460)
* intial working

* intial working with test

* return tests

* fix interval and add monthly test

* add in person_id join

* small restyle

* postgres lifecycle working

* add action handling to postgres query

* add action handling and tests

* fix typing

* visualizing with temp params and added negatives for dormant

* frontend

* fix intervals

* remove unnecessary import

* add person enppoint

* add person endpoint and tests

* add next

* add pagianted test

* fix types

* add frontend logic'

* fixed date range

* disable compare filter on lifecycle

* add diasbled to chartfilter

* return class to personviewset

* added constant

* fix distinct_id and new event within period condition

* replace people queries and fix ch query too

* DRY

* fix null states

* comparefilter nullstate

* add wrapper back to endpoint

* fix datetime formatting

* remove extra stack flag

* reduce filters when it's lifecycle and replace constants

* add default for lifecycle into trendlogic
2020-12-02 16:53:06 +01:00
Karl-Aksel Puulmann
a451e149c6
Add support for median, p90, p95 and p99 math functions (#2630)
* Add support for median, p90, p95 and p99 math functions

Related issue: #2584

* Update tests to reflect quantile functions are not exact in ch
2020-12-02 17:22:45 +02:00
Michael Matloka
b5f7bbf232
Add /api/projects/:project_id/actions/ (#2616) 2020-12-02 06:40:47 +01:00
Eric Duong
98932d0c3a
1936 retention graph (#2578)
* intiial graph on PG

* initial frontend

* fix filtering and types

* clickhouse working

* change reference date

* fix label and defaults

* fix multiple event retention

* working clickhouse and postgres for multiple events

* remove final

* remove sql parse

* add inital person query

* working clickhouse person query and return testrunner

* implement postgres retrieve person

* add endpoint and pagianted test

* click on datapoint works

* fix params

* fix action handling

* people paginated backend

* finished paginatino frontend logic

* fix variable naming for pg

* working first time persons for clickhouse

* first time people working for pg

* dont replace if recurring

* run again

* fix import

* refactor filter to retention specific and move postgres

* convert clickhouse to use new filter

* move imports

* fix types

* fix types

* fix initial load

* adding to dashboard works

* fixed defaults

* add test for dates

* fix insight panel

* add seert

* fix import

* fix date select for people

* fix chartfilterlogic

* remove unneded comment and import

* split up retention query
2020-12-01 10:15:50 -05:00
Eric Duong
5ddfe6c30d
Revert "Retention graph and multiple event fix (#2534)" (#2577)
This reverts commit bb076eedad.
2020-11-30 16:40:22 -05:00
Eric Duong
bb076eedad
Retention graph and multiple event fix (#2534)
* intiial graph on PG

* initial frontend

* fix filtering and types

* clickhouse working

* change reference date

* fix label and defaults

* fix multiple event retention

* working clickhouse and postgres for multiple events

* remove final

* remove sql parse

* add inital person query

* working clickhouse person query and return testrunner

* implement postgres retrieve person

* add endpoint and pagianted test

* click on datapoint works

* fix params

* fix action handling

* people paginated backend

* finished paginatino frontend logic

* fix variable naming for pg

* working first time persons for clickhouse

* first time people working for pg

* dont replace if recurring

* run again

* fix import

* refactor filter to retention specific and move postgres

* convert clickhouse to use new filter

* move imports

* fix types

* fix types

* fix initial load

* adding to dashboard works

* fixed defaults

* add test for dates

* fix insight panel

* add seert

* fix import

* fix date select for people

* fix chartfilterlogic
2020-11-30 16:09:23 -05:00
Eric Duong
3f5b76cb8c
DAU breakdown and filter bug (#2575)
* add table name

* add test
2020-11-30 15:51:12 -05:00
Tim Glaser
9823e38163
Move away from ewap to speed up queries (#2540)
* Move away from ewap to speed up queries

* Lower limit to 10
2020-11-30 11:22:31 +01:00
Paolo D'Amico
6426408c6a
remove legacy cloud billing info from ee (#2537) 2020-11-29 11:09:25 +00:00
Karl-Aksel Puulmann
940b3d1b65
Don't show session recording play button if no FullSnapshot rrweb events (#2547)
This avoids showing fully broken recordings to users. We're working on
reducing the number of errors.
2020-11-27 13:36:24 +01:00
Tim Glaser
260b999093
Revert "Retention graph and multiple event fix (#2405)" (#2532)
This reverts commit e90857900e.
2020-11-26 15:52:45 +01:00
Eric Duong
e90857900e
Retention graph and multiple event fix (#2405)
* intiial graph on PG

* initial frontend

* fix filtering and types

* clickhouse working

* change reference date

* fix label and defaults

* fix multiple event retention

* working clickhouse and postgres for multiple events

* remove final

* remove sql parse

* add inital person query

* working clickhouse person query and return testrunner

* implement postgres retrieve person

* add endpoint and pagianted test

* click on datapoint works

* fix params

* fix action handling

* people paginated backend

* finished paginatino frontend logic

* fix variable naming for pg

* working first time persons for clickhouse

* first time people working for pg

* dont replace if recurring

* run again

* fix import

* refactor filter to retention specific and move postgres

* convert clickhouse to use new filter

* move imports

* fix types

* fix types

* fix initial load

* adding to dashboard works

* fixed defaults

* add test for dates

* fix insight panel

* add seert

* fix import

* fix date select for people
2020-11-26 15:18:37 +01:00
Karl-Aksel Puulmann
38f8364d7a
Return person data together with session recording (#2528)
* Return person data together with session

This will be used in the new session recording page

* Add names to anonymous types
2020-11-26 16:06:50 +02:00
Eric Duong
ec45964d0a
Fix prop filter not formatting properly (#2386)
* force parsed

* robust fix

* remove sql parse
2020-11-26 13:48:17 +01:00
Karl-Aksel Puulmann
905d2e57c6
Hotfix: fix session recordings (#2502) 2020-11-25 12:13:55 +02:00
Michael Matloka
96e4ee8512
Nest endpoints of project-based models under /api/project/ – LITE (#2485)
* Nest endpoints under /project/ with StructuredViewSetMixin

* Rewrite URLs

* isort

* Update utils.py

* Fix errors

* Fix almoast all the errors

Last left to do: shared dashboards and permission classes.

* isort

* Adjust for master

* Add compatbility with shared dashboards

* Debug ClickHouse

* Remove some # type: ignores

* Simplify CursorPagination

* Move test base from posthog.api.test to posthog.test

* Improve API structure

* Bring back legacy endpoints

* Fix legacy compatibility

* Fix bugs and typing

* isort

* Fix hooks test

* Try fixing errors

* Fix oversight

* isort

* Fix problems

* isort

* Be more tolerant

* Fix naming and remove redundant code

* Fix imports

* Update deleteWithUndo

* Roll back

* Roll back more

* Update .gitignore

* Rollll back

* Rollllllll

* back

* Betterify

* Address feedback
2020-11-24 23:26:28 +01:00
Tim Glaser
f47e8b97ae
Combine events and actions into select box (#2394)
* WIP combine events and actions

* Alignment

* Keyboard navigation select box

* Fix enter and other small issues

* add icons

* fix renderinfo

* Hide scrollbar

* Refactor

* Fix cypress test

* fix cypress test
2020-11-24 18:28:44 +01:00
Tim Glaser
ac24aa903e
Revert "Event usage split clickhouse queries (#2388)" (#2469)
This reverts commit 7e343c2a89.
2020-11-23 13:31:02 +01:00
Tim Glaser
7e343c2a89
Event usage split clickhouse queries (#2388)
* Event usage split clickhouse queries

* Ignore

* Remove unnecessary code

* Use Karl's refactor
2020-11-23 12:40:09 +01:00
Michael Matloka
d24783ece6
Org/projects UX enhancements (#2145)
* Improve member removal UX

* Add Delete Project

* Constrain project deletion in the frontend

* Improve no org

* Make minor fixes

* Bring back changes

* Add member level changing

* Update test_organization_members.py

* Fix org leaving edge case

* Polish Project Settings UI

* Allow full project deletion

* Fix dynamic page title edge case

* Mymymypy

* Remove console.log

* Remove obsolete test

* Fix API constraints

* Fix invite creation validation case

* Add cloud/self-hosted specificity to premium feature guarding

* Improve user setting logic in case of error

* Use Teammates instead of Team Members

* Improve org/project creation UX

* Add organization ownership

* Add migration

* Fix redirecting to org creation

* Improve clipboard toast consistency

* isort

* Update tests

* isort

* Update test_organization.py

* Fix typing (somewhat)

* Explain passing on ownership

* Address feedback

* Reverse guardPremiumFeature return

* Fix typing

* Fix test

* Fix last typing

* Fix merge conflict resolution

* Remove unused code
2020-11-23 10:53:45 +01:00
Marius Andra
6906c86a5c
Use isoformat() for dates in Kafka WAL (otherwise playback results in errors); add back sentry instrumentation (#2396) 2020-11-20 14:47:03 +01:00
James Greenhill
e32ba846ed
Revert "Add alt_host for graceful failover to our secondary clickhouse node (#2407)" (#2446)
This reverts commit 751b47c73f.
2020-11-19 21:45:58 -08:00
James Greenhill
751b47c73f
Add alt_host for graceful failover to our secondary clickhouse node (#2407) 2020-11-19 21:18:46 -08:00
Tim Glaser
7bd0ea7039
Fix fetch single event clickhouse (#2436) 2020-11-19 11:27:29 +01:00
Tim Glaser
78c2ae74ac
Fix filtering is not bool (#2435)
* Fix filtering is not bool

* don't commit settings
2020-11-18 19:30:16 +01:00
Tim Glaser
b41381c149
Fix empty actions in cohorts (#2411)
* Fix empty actions in cohorts

* Dont show calculating in clickhouse

* Fix
2020-11-18 16:30:29 +01:00