0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-12-01 04:12:23 +01:00
Commit Graph

2092 Commits

Author SHA1 Message Date
Marius Andra
353ecda5b1
add session recording stats to instance status page (#2861) 2021-01-06 11:11:41 +01:00
Marius Andra
71b022b8cb
fix bug from query-selector-shadow-dom update (#2856) 2021-01-05 19:48:27 +01:00
Karl-Aksel Puulmann
947dec4b2c
Kill N+1 in sessions_list (#2850) 2021-01-05 17:36:21 +02:00
Karl-Aksel Puulmann
157003e2ab
Tests, fix for createActionFromEvent, clean up sessions query (#2851)
* Remove a dead TODO

This was solved in PR  #1849

* Make sessions query more readable

* Remove dead code from sessions

* Make it possible to test with imports in jest

* Fix a bug and add tests to createActionFromEvent

I added tests when initially solving #2848, but the original solution
had to be scrapped. The tests are still valid though
2021-01-05 17:11:24 +02:00
Karl-Aksel Puulmann
e03477b7d4
Index session recording events by team_id, timestamp (#2853)
* Index session recording events by team_id, timestamp

This speeds main session recording query up significantly. Solves #2739

----

Measurements

Tested this on a semi-large self-hosted instance.

Pre-index:

```
| QUERY PLAN                                                                                                                                                                                                    |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Subquery Scan on p  (cost=207673.55..207673.60 rows=1 width=97) (actual time=22200.985..22200.985 rows=0 loops=1)                                                                                             |
|   ->  GroupAggregate  (cost=207673.55..207673.59 rows=1 width=105) (actual time=22200.984..22200.984 rows=0 loops=1)                                                                                          |
|         Group Key: posthog_sessionrecordingevent.distinct_id, posthog_sessionrecordingevent.session_id                                                                                                        |
|         Filter: (count(*) FILTER (WHERE ((posthog_sessionrecordingevent.snapshot_data ->> 'type'::text) = '2'::text)) > 0)                                                                                    |
|         Rows Removed by Filter: 20                                                                                                                                                                            |
|         ->  Sort  (cost=207673.55..207673.56 rows=1 width=468) (actual time=22185.713..22186.010 rows=4937 loops=1)                                                                                           |
|               Sort Key: posthog_sessionrecordingevent.distinct_id, posthog_sessionrecordingevent.session_id                                                                                                   |
|               Sort Method: quicksort  Memory: 2932kB                                                                                                                                                          |
|               ->  Index Scan using posthog_ses_team_id_46392f_idx21 on posthog_sessionrecordingevent  (cost=0.56..207673.54 rows=1 width=468) (actual time=127.291..22181.190 rows=4937 loops=1)              |
|                     Index Cond: ((team_id = 1) AND ("timestamp" >= '2020-12-01 10:37:21.726776+00'::timestamp with time zone) AND ("timestamp" <= '2021-01-05 11:30:47.010794+00'::timestamp with time zone)) |
| Planning Time: 0.195 ms                                                                                                                                                                                       |
| Execution Time: 22201.031 ms                                                                                                                                                                                  |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```

After index:
```
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                                                                                                                                    |
|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Subquery Scan on p  (cost=8.46..8.51 rows=1 width=97) (actual time=20.420..20.420 rows=0 loops=1)                                                                                                             |
|   ->  GroupAggregate  (cost=8.46..8.50 rows=1 width=105) (actual time=20.419..20.419 rows=0 loops=1)                                                                                                          |
|         Group Key: posthog_sessionrecordingevent.distinct_id, posthog_sessionrecordingevent.session_id                                                                                                        |
|         Filter: (count(*) FILTER (WHERE ((posthog_sessionrecordingevent.snapshot_data ->> 'type'::text) = '2'::text)) > 0)                                                                                    |
|         Rows Removed by Filter: 20                                                                                                                                                                            |
|         ->  Sort  (cost=8.46..8.47 rows=1 width=468) (actual time=13.695..14.078 rows=4937 loops=1)                                                                                                           |
|               Sort Key: posthog_sessionrecordingevent.distinct_id, posthog_sessionrecordingevent.session_id                                                                                                   |
|               Sort Method: quicksort  Memory: 2921kB                                                                                                                                                          |
|               ->  Index Scan using posthog_ses_team_id_46392f_idx24 on posthog_sessionrecordingevent  (cost=0.43..8.45 rows=1 width=468) (actual time=0.020..2.295 rows=4937 loops=1)                         |
|                     Index Cond: ((team_id = 1) AND ("timestamp" >= '2020-12-01 10:37:21.726776+00'::timestamp with time zone) AND ("timestamp" <= '2021-01-05 11:30:47.010794+00'::timestamp with time zone)) |
| Planning Time: 15.432 ms                                                                                                                                                                                      |
| Execution Time: 22.521 ms                                                                                                                                                                                     |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```

* make linter happy
2021-01-05 17:10:21 +02:00
Marius Andra
22ef7cec6f
Custom NPM packages under @posthog (#2854)
* move some packages out of the @mariusandra/ npm namespace

* Fix yarn.lock

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-01-05 14:42:19 +01:00
Eric Duong
4ece4ce3e8
Major filter refactor (#2736)
* split params into mixins

* add typing

* split func

* dedicated paths filter

* make basefilter

* restructure directories

* move prop mixin

* implement cached_property and fix circular import

* remove unused imports

* stickiness filter converted

* temp patch for breakdown

* correct naming for derived mixins in paths

* fix types

* fix types

* fix casing

* fix breakdown arg

* merge master

* add to_dict

* DRY to_dict

* scoped to_dict ability

* fix types

* add missing fields

* refactor session filter

* remove unused imports

* remove unused imports

* add stickiness filter test

* fix dict for pathfilter

* properly load strs

* add default

* standardize loading

* retention filter separate

* add update filter to retention

* change back decorator

* don't allow setting on filter date_from date_to

* remove seconds on default timestamp

* remove derived values

* fix date formatting

* fix typo
2021-01-05 12:15:24 +01:00
Paolo D'Amico
e169a74346
Support different Reply-To header for most emails (#2846)
* adds reply-to support for emails

* add missing from config
2021-01-05 10:48:20 +01:00
Eric Duong
6241a2b8a3
Quick fix to limit time range on event prop query (#2844)
* add range limit on props

* add exception handler

* fix time limit
2021-01-04 14:02:38 -05:00
Tim Glaser
b2a9e36444
Fix relative dates filters (#2843) 2021-01-04 15:09:52 +01:00
Michael Matloka
b579a67fee
Clean up bookmarklet (#2812) 2021-01-04 14:27:42 +01:00
Michael Matloka
ff53d2ed09
Generate event_uuid slightly earlier for plugin server idempotency (#2796) 2021-01-04 14:27:24 +01:00
Michael Matloka
e284a29757
Refactor useEventListener to TypeScript (#2841) 2021-01-04 14:26:37 +01:00
Karl-Aksel Puulmann
4a3b154909
Allow * in action selector (#2820)
* Refactor filtering by selector tests

* Support action selector * in clickhouse actions

Fixes https://github.com/PostHog/posthog/issues/2819
2021-01-04 11:35:23 +02:00
dependabot[bot]
51b291602e
Bump node-notifier from 8.0.0 to 8.0.1 (#2837)
Bumps [node-notifier](https://github.com/mikaelbr/node-notifier) from 8.0.0 to 8.0.1.
- [Release notes](https://github.com/mikaelbr/node-notifier/releases)
- [Changelog](https://github.com/mikaelbr/node-notifier/blob/v8.0.1/CHANGELOG.md)
- [Commits](https://github.com/mikaelbr/node-notifier/compare/v8.0.0...v8.0.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2020-12-23 12:56:10 +01:00
Marius Andra
39239c2780
Upgrade Plugin Server (#2816)
* use KAFKA_ENABLED key

* 0.6.1 plugin server
2020-12-21 21:57:29 +01:00
Karl-Aksel Puulmann
36c6bf8e30
Fix retention filter dashboard items (#2815)
Fixes https://github.com/PostHog/posthog/issues/2813
2020-12-18 11:18:58 +02:00
Karl-Aksel Puulmann
519e28cf5e
Fix disabling server-side cursors (#2814)
From
https://docs.djangoproject.com/en/3.1/ref/settings/#disable-server-side-cursors
seems like we were setting this wrong.
2020-12-18 10:47:41 +02:00
Karl-Aksel Puulmann
67e431f990
Do not use SessionsFilter for insights sessions (#2807)
This is a bit confusing (my bad) - both insights and sessions list page
used to run under the same code path, but no more. SessionsFilter is for
sessions list.

Will fix and rename it once I get my open PR in :)
2020-12-17 10:34:30 +01:00
James Greenhill
5419370f79
Clean up prod dockerfile, remove dupe calls (#2806) 2020-12-16 20:48:15 -08:00
James Greenhill
0f124d5feb
Make testing using docker a bit easier with docker-compose for ee (#2800) 2020-12-16 20:35:05 -08:00
James Greenhill
3d0413bdcb
Verbose Yarn build on plugin server build (#2805) 2020-12-16 18:06:56 -08:00
James Greenhill
d93ae873a6
Make building with yarn more verbose for debugging (#2804) 2020-12-16 17:14:57 -08:00
James Greenhill
a8de2ac587
plugins debug (#2803)
* checkin

* Don't run yarn silently for plugin-server
2020-12-16 16:35:19 -08:00
Michael Matloka
c9664e94fb
Issue templates plus (#2681)
* Update issue templates format

* Create imperfection_report.md

* Create performance_issue_report.md

* Rewrite Bug report `about`

* Address feedback

* Delete imperfection_report.md
2020-12-17 00:49:37 +01:00
Marius Andra
c191fef86b
Ignore plugins optional deps (#2801)
* ignore optional deps when installing plugin-server

* --ignore-optional in docker
2020-12-16 23:59:40 +01:00
Paolo D'Amico
febaed2340
Feature flag Papercups (#2799)
* feature flag papercups

* Use @papercups-io/chat-widget instead of weird useEffect hacks

* Rename basicComponents to essentialElements

* Add posthog_domain to Papercups metadata

* Actually rename basicComponents to essentialElements

Co-authored-by: Michael Matloka <dev@twixes.com>
2020-12-16 21:42:32 +01:00
Marius Andra
29824fbfed
Fix Docker Build (#2797)
* add build-essential

* also purge it
2020-12-16 17:28:30 +01:00
Karl-Aksel Puulmann
7cbcf07d58
WIP: Try to fix error with DAU breakdown of user properties (#2793)
* add tests and fix aggregated value

* double check when testing

* double check when testing

* typing

* fix tests for ee

* WIP: Try to fix error with DAU breakdown of user properties

This still fails a test, but fixes the original issue: two joins against
persons table were being made.

https://sentry.io/organizations/posthog/issues/2094211477/?project=1899813&query=is%3Aunresolved

* use annotate

* remove extra annotation

* change func name

* WIP: Try to fix error with DAU breakdown of user properties

This still fails a test, but fixes the original issue: two joins against
persons table were being made.

https://sentry.io/organizations/posthog/issues/2094211477/?project=1899813&query=is%3Aunresolved

* change event_join handling

Co-authored-by: Eric <eeoneric@gmail.com>
2020-12-16 18:23:50 +02:00
Michael Matloka
818576a4d6
Allow Plugins on Cloud (#2783)
* Allow Plugins configuration on Cloud

* Generate UUID in log_event for idempotency

* Make event UUID common for both EE event processing pipelines

* Restore frontend from master for changed direction

* Fix _process_event_ee

* Whitelist PostHog Inc. and KeaJS for Plugins on Cloud

* Test Plugins on Cloud organization whitelisting

* Fix FOSS

* Revert UUID changes (another PR)

* Update posthog-plugin-server to 0.6.0-beta
2020-12-16 16:48:41 +01:00
Tim Glaser
3f7e95d14a
Improved insights history (#2745)
* Deprecate dashboard item type and move to display

* Mypy rerors

* fix test

* fix

* Fix test

* Fix another test

* WIP save history refactor

* Remove determineInsightType

* Progress

* Get rid of RetentionTable display types

* Sync all together

* Fix update dashboard

* Improved saving

* More bugfixes

* Fix insight caching and filters

* Fix insights

* Fix tests

* Fix import

* Fix saving issues

* Don't duplicate

* fix

* Use session instead of session_type everywhere for consistency

* Remove prints

* Use get_filter

* Fix retention filters

* Fix UI issues

Co-authored-by: Michael Matloka <dev@twixes.com>
2020-12-16 16:37:55 +01:00
Eric Duong
2a679ed353
Fix breakdown aggregated value (#2785)
* add tests and fix aggregated value

* double check when testing

* double check when testing

* typing

* fix tests for ee

* use annotate

* remove extra annotation

* change func name
2020-12-16 16:46:56 +02:00
Michael Matloka
259e801de6
Whitelist plugins per organization on Cloud (#2791)
* Fix unix-dgram build

* Add per-organization whitelisting of plugins API on Cloud

* Add migration

* Update can_install_plugins_via_api calls in serializer

* Remove unused type: ignore

* Update PluginSerializer for organization_id

* Make Plugin.organization nullable

* Fix can_install_plugins_via_api

* Update pluginsLogic.ts

* Adjust can_configure_plugins_via_api for organization

* Don't include organization_id in serializer

* Set default Plugin.organization to first Organization in instance

* Fix PluginConfigSerializer.create

* Test that other orgs' plugins can't be accessed (as if they didn't exist, 404)

* fix typo

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2020-12-16 14:54:30 +01:00
Karl-Aksel Puulmann
05f66f99a1
Create a bookmarklet for demoing posthog (#2774)
* add bookmarklet instructions to setup

* minor change

Co-authored-by: yakkomajuri <yakko.majuri@gmail.com>
2020-12-16 10:34:23 -03:00
Yakko Majuri
b70034dc0d
Update CHANGELOG.md (#2788) 2020-12-16 07:35:09 -03:00
Paolo D'Amico
cc58fcace2
disable event property filtering (#2784) 2020-12-16 12:01:53 +02:00
James Greenhill
2f9f3fe05a
Fixes for differences between terraform and GA task-definition (#2786) 2020-12-15 18:34:01 -08:00
Marius Andra
098343cbee
Deploy Plugins to AWS (#2755)
* create plugins task

* add plugins to container

* remove env vars that are probably not needed

* bump size of node for plugins

* don't run plugin server on worker

Co-authored-by: James Greenhill <fuziontech@gmail.com>
2020-12-15 18:10:00 -08:00
Eric Duong
17253de766
fix pie aggregate value (#2781) 2020-12-15 17:11:22 -05:00
Yakko Majuri
4aea278e70
Release 1.19.0 (#2780)
* Hide bar from retention (#2770)

(cherry picked from commit bf506e3d0b)

* Bump version 1.19.0

* fix gh broken link
2020-12-15 17:35:05 -03:00
James Greenhill
2935a65dc5
MV -> View for events_with_array_props_view and remove EVENT_PROP_TABLE_SQL (#2766)
* MV -> View for events_with_array_props_view

* sort

* convert another mv into view because mv's are not triggered by views

* pedantic - no mat in naming

* remove unused tables
2020-12-15 10:59:28 -08:00
Eric Duong
826182b8d2
Fix total aggregate values (#2723)
* postgres working

* change result attr name

* adhoc solution for clickhouse query

* add breakdown filter test

* adhoc breakdown solution

* add breakdown default

* fix frontend logic

* breakdown refactoring

* reshape breakdown functions

* remove print

* remove dump.rdb

* replace magic string with constant
2020-12-15 18:37:25 +01:00
Paolo D'Amico
3b6a08da82
Not-so-big play button (#2744)
Co-authored-by: Karl-Aksel Puulmann <oxymaccy@gmail.com>
2020-12-15 11:02:08 -06:00
Eric Duong
5845e3ec8b
Move lifecycle tests to separate file (#2765)
* put lifecycle in separate file

* use list
2020-12-15 16:05:38 +01:00
Paolo D'Amico
6d69b029e0
Navigation 1775 improvements & fixes (#2763)
* add backTo in new navigation

* fix #2541

* add billing & licenses link

* add annotations
2020-12-15 16:48:14 +02:00
Yakko Majuri
0b01bb694e
Honor next URL when logged out (#2757)
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-12-15 08:25:15 -06:00
Yakko Majuri
bf506e3d0b
Hide bar from retention (#2770) 2020-12-15 10:15:56 -03:00
Karl-Aksel Puulmann
fe8b5e0dd7
Optimize breaking down by event property value (#2767)
* Optimize breaking down by event property value

Just getting the top values took 10+ seconds previously for our team,
now down to 2, and the breakdown query went from 6s -> 2s.

* solve typecheck issue
2020-12-15 13:11:35 +02:00
Yakko Majuri
d9f9545b18
Fix Identified persons pagination (#2756) 2020-12-14 15:48:02 -06:00
Marius Andra
f3e4cad700
Plugin editor (#2743)
* fix tsc error

* add plugin source, create plugin source version model, add plugin_type

* can create and save source plugins via the api

* make empty source plugins

* message if no config options available

* different image or tag for source plugins

* fix some types

* second drawer

* add fields into drawer

* add monaco editor

* refactor drawer

* save plugin and must be json

* close tab on save

* fix default code

* upgrade to plugin server 0.5.0 - with scheduled plugins

* less height

* upgrade to plugin server 0.5.1 - with posthog.capture

* remove minimap from editor

* upgrade to 0.5.1

* use enum

* reduce quirk

* must enter a name/url

* use the source

* simpler text

* sync image

* add link to docs

* add a link to the documentation inside the source code drawer

* text / ux simplifications

* nicer first plugin experience

* nicer django model choices
2020-12-14 17:11:29 +01:00