0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-22 08:40:03 +01:00
Commit Graph

124 Commits

Author SHA1 Message Date
Sam Winslow
60cbf2e7c1
Trigram fuzzy search for property definitions (#5039)
* Trigram fuzzy search for property definitions

* support empty search param

* Only use pg_trgm search if pg extension is installed

* suppress exceptions if migration does not succeed

* checks for index

* troubleshooting

* wip: change migration number; strip out conditionals

* wip: copy code from #4110

* remove is_pg_trgm_installed

* try ngram search on CH

* add AND before ngram

* correct some check failures

* remove ordering

* remove ordering from tests

* fix ordering

* rewrite queries for psql, add test bypass for EE features

* remove ordering filter bc of bugs

* remove test code

* check if isinstance of RawQuerySet

* cleanup sql

* add ordering

* typing

* test for ee prop definition

* adjustments to test_ee_property_definition

* add test_ee_event_definition

* restore this prop

* convert to simple term filter

* use term search in EE

* fix EE tests

* fix non-EE tests

* fix test

* fix filter types

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-07-20 14:58:16 +02:00
Paolo D'Amico
1b190e89cf
SSO domain whitelist (#5111) 2021-07-15 18:20:43 -05:00
Tim Glaser
208d201bba
Remove accidental breakdown on funnels (#5106)
* Remove accidental breakdown on funnels

* fix
2021-07-13 18:17:54 +02:00
Yakko Majuri
6c9bb2db0f
Update token gen (#4832)
* use base36 for random tokens

* Use prefix + base62

* add migration

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-06-23 13:19:05 -03:00
Yakko Majuri
a71a1eac96
add metrics to plugin table (#4783) 2021-06-22 11:07:05 -03:00
Li Yi Yu
7546e3ea0a
Event properties taxonomy (#4486)
* add description and tag fields to event and property definitions

* set up description and tagging on models

* frontend functionality for description editing

* connect backend and kea logic for description editing

* update event and property definitions model and migration

* delete set null instead of cascade

* migration merge fix

* add owner column

* undo posthog event property taxonomy migrations

* set up definitions on enterprise level

* allow enterprise definitions description editing

* fix licensing conditions and add tests

* proper get and update methods for the multi inheritance table and new column fields for enterprise event model

* check for license to separate routes

* migrate existing definitions to ee definitions tables and render ee vs non-ee definition views based on existing feature conditional

* all the working backend updates

* updated tests

* frontend fixes and linting updates

* feature flag it
2021-06-03 09:22:16 -04:00
Paolo D'Amico
1b0240d7fa
Insights short share URL (#4513) 2021-06-01 07:18:19 -07:00
Michael Matloka
eaf2459724
Save Organization.available_features as a DB column (#4426)
* Save Organization.available_features as a DB column

* `update_available_features()` before an organization is created too

* Run the task half past the hour

* Adjust tests and fix available_features sync on start

* Remove redundant null=False

* Fix `Organization.update_available_features()`

* Run Cloud tests in respect to `posthog-cloud` `4426-fix` branch

* Revert "Run Cloud tests in respect to `posthog-cloud` `4426-fix` branch"

This reverts commit 421e8541b3.
2021-05-24 22:17:01 +02:00
Karl-Aksel Puulmann
030529cfb3
Show internal metrics on /instance/status page (#4366)
* Extract function for capture

* Add capability to capture internal metrics

- This is behind an environment variable, we'll turn this on by default
on clickhouse-based installation.
- A special Org/Team gets created for this lazily, which is not accessible for
anyone. This makes avoiding issues with stats/billing/properties and easier.

* Capture some useful metrics into posthog

* Set up an unique constraint for for_internal_metrics organization

This avoids some nasty race conditions

* internal_metrics fixes

* Extract component OverviewTab

* New tabbing system for system status page

* Embed ClickhouseTab

* Generate a dashboard for internal metrics on demand

* More metrics

* Add more internal metrics, add tests

* Fix migration files

* Adjust copy

* Update task definitions

* Fix a typing issue

* Rename a component

* Migration fixup

* Try reset patch after test ends

* Clear lru_cache between tests
2021-05-20 13:42:39 +03:00
Neil Kakkar
7be1b0ef92
Add Capabilities to Plugin model (#4371)
* Add capabilities to plugin model

* update tests
2021-05-18 11:46:01 +01:00
Tim Glaser
1ae890fbe2
1534 configurable columns person events table (#4141)
Co-authored-by: Mohit Gupta <mohitgupta3mail@gmail.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
Co-authored-by: Sam Winslow <sammywinslow@gmail.com>
2021-05-13 14:13:54 -07:00
Michael Matloka
a126e73cb7
Preinstall plugins (#4106)
* Refactor plugin installation and reloading logic

* Preinstall and autoenable plugins (only GeoIP one for now)

* Fix PluginManager.install issues

* Add tests

* Disable plugin preinstallation on Cloud

* Check plugins access level before preinstallation

* Rename field `preinstalled` to `is_preinstalled`

* Adjust tests for Cloud v. self-hosted

* Fix test_preinstalled_are_autoenabled

* Ensure install plugins access level for preinstallation tests

* Bump migration

* Bump migration

* Add and use Plugin.get_default_config, with tests

* Refactor private PluginConfigSerializer methods out of the class

* Skip preinstallation of plugin if it couldn't be done (e.g. airgapped)

* Don't hit GitHub API in tests

* Satisfy mypy

* Fix a test on Cloud CI
2021-05-07 03:02:04 +02:00
Michael Matloka
fb0bd54af6
Increase Element model varchar limits again (#3926)
* Revert "Revert "Increase Element model varchar limits (#3912)" (#3923)"

This reverts commit e30681bade.

* Use RunSQL to avoid ORM crap

* Fix attr_class being converted from varchar[] to varchar

* Bump migration

* Simplify migration and remove attr_class alter due to locking

* Bump migration
2021-05-06 17:11:37 +00:00
Paolo D'Amico
fee0b1035d
Fix lifecycle insights store (#4223)
* lifecycle migration

* update latest migrations

Co-authored-by: eric <eeoneric@gmail.com>
2021-05-06 11:27:42 -04:00
Karl-Aksel Puulmann
9d8b61f2e1
Fix conflicting migrations (#4225)
Error:

```
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0147_fix_stickiness_dashboard_items, 0147_plugin_logs in posthog).
To fix them run 'python manage.py makemigrations --merge'
```
2021-05-06 08:31:57 +00:00
Michael Matloka
e96f95ef5a
Plugin log entries (#3482)
* Add Postgres model PluginLogEntry

* Add equivalent PluginLogEntry to Kafka+ClickHouse

* Add migration

* Add PluginLogEntry.Type.LOG & make PluginLogEntry.message a TextField

* Update 0130_pluginlogentry.py

* Add PluginLogEntry.instance_id

* Update migration

* Update migration

* Add plugin log entries API

* Test plugin log entries DB fetching

* Add PluginLogs component prototype

* Fix API

* Improve PluginLogs component

* Remove almost unused plugin Feedback button

* Update migration

* Fixed typing

* Fix org permission error test asserts

* Fix plugin log entry tests

* Fix CH plugin log entry timestamp string

* Update CH test_plugin_log_entry.py

* Fix plugin log entry tests across PG/CH

* Satisfy mypy

* Add search and limit to plugin log entry API

* Send team_id in plugin config API

* Rework plugin logs UI

* Add plugin config team ID in tests

* Add plugin config team ID in tests actually

* Fix code quality

* Make logs plugin config-based

* Fix CH queries

* Fix typing

* Improve UX and fix things

* Polish plugin logs logic

* Update migration

* Add Celery task to delete old plugin logs

* Fix UX bug with loading more plugin logs

* Fix missing import

* Remove OrganizationMemberPermissions message change

* Make mypy happy

* Add PluginLogEntry.is_system

* Optimize CH plugin_log_entires PARTITION/ORDER

* Increment migration

* Adjust plugin logs drawer display

* Fix plugin_log_factory_ch

* Fix plugin_log_factory_ch fix

* Replace PluginLogEntry.is_system with source

* Adjust PluginLogEntrySerializer

* Update CH fetch_plugin_log_entries

* Make kea-typegen happy
2021-05-06 10:54:32 +03:00
Marius Andra
ef1a5470a9
Migration to sync team event properties (#4101)
* migration to add delayed jobs to sync team event properties

* remove .delay

* update latest migrations

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-04-26 13:16:48 +03:00
Paolo D'Amico
072a6c1beb
Refactor event & properties for async server-side loading (#4070)
* remove event & properties endpoint from team endpoint

* event definition model

* sync events from Team with EventDefinition

* fix typing

* missing commit

* event definition API

* add querying capabilities

* property definition model

* sync property definitions

* temp: bring back event properties to Team model to split PR

* PropertyDefinition API + tests

* fix tests for Python 3.7 & more deprecation notices

* default values to 0 instead of null

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-04-22 08:42:01 +00:00
Paolo D'Amico
7ea9eb692c
Update Django to 3.1.8 (and related) (#4007) 2021-04-19 12:59:10 -07:00
Tim Glaser
5cfbc49cc3
Add clickhouse and max users to license (#3918)
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-04-14 08:45:39 -07:00
Michael Matloka
e30681bade
Revert "Increase Element model varchar limits (#3912)" (#3923) 2021-04-08 21:24:03 +02:00
Michael Matloka
305e5f7210
Increase Element model varchar limits (#3912) 2021-04-08 20:17:29 +02:00
Paolo D'Amico
9865bbc3c1
User V2 API endpoint (#3855)
* initial refactor

* simple serializer for teams

* basic team in front-end

* use basicTeam where applicable

* refactor UserSerializer to UserBasicSerializer

* abstract UserBasicSerializer

* add uuid to users

* move event properties methods to Team model

* basic V2 user endpoint

* update current user

* update current team and org

* post-merge fix

* fix load logic

* change password

* add some extra tests

* undo all frontend changes from this PR

* remove last frontend change

* update preflight tests

* remove team basic fetch

* fix foss tests

* adjust migration & add distinct ID to payload

* add deprecation notice

* add uuid to UserBasic serializer & update memberships

* add available features to org serializer

* address feedback

* endpoint -> `users/@me/`

* address feedback

* Use UUIDClassicModel abstract model when adding uuid field

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-04-05 21:42:31 +02:00
Michael Matloka
2c8b63b68a
Fix Team.data_attributes default (#3846)
* Fix Team.data_attributes default

* Add test
2021-04-01 21:06:45 +02:00
Marius Andra
581a617a95
Add index to events.created_at (#3767)
* add index to events.created_at

* convert to concurrent index
2021-03-30 10:43:24 +02:00
Marius Andra
7ed7c4c649
Custom Data Attributes in the toolbar (#3761)
* add `important_data_attributes`

* fix undefined bug

* important data attributes in setup

* add dashes

* remove dead code

* scarier exampe

* support multiple data attributes in the toolbar

* update to simmer that supports wildcards

* rename "important_data_attributes" -> "data_attributes", store as a json array

* update ts blacklist

* another one

* use a tag select for data attributes

* remove needless string serialization

* implement text feedback

* data attribute setup style improvement

* better default handling

* fix out of sync migration
2021-03-30 10:28:57 +02:00
Paolo D'Amico
da61c55666
Description & tagging on dashboards (#3756) 2021-03-26 17:38:13 -07:00
Paolo D'Amico
136a021d1a
Feature Flags Revamp (UX & Code) (#3675) 2021-03-23 14:34:48 -07:00
Paolo D'Amico
a8838701aa
Timezone handling UX (#3698)
Co-authored-by: Tim Glaser <tim@glsr.nl>
Co-authored-by: Michael Matloka <dev@twixes.com>
2021-03-23 12:54:33 -07:00
Michael Matloka
ac5fa0686b
Allow for non-plugin config-keyed plugin attachments (#3732) 2021-03-23 18:46:56 +01:00
Michael Matloka
828e0fd5c2
Solidify plugins on Cloud rollout (#3706)
* Better handle extra info buttons during plugin reordering

* Fix UI bug in first time plugin enabling

* Give very new Cloud org config access to plugins

* Add migration
2021-03-19 14:25:09 +01:00
Michael Matloka
3c571f1312
Different async action mapping approach (#3678)
* Disable useless InsecureRequestWarning

* Disallow ASYNC_EVENT_ACTION_MAPPING on CH and make it default otherwise

* Return early from calculate_actions_from_last_calculation if CH

* Optimize Action.calculate_events and fire hooks from it

* Run calculate_event_action_mappings every 30 s instead of 5 min

* Optimize post_event_to_webhook thanks to new actions approach

* Add Event.site_url field for reworked Postgres webhooks

* Adjust tests by using sync calls to actions calculation

* Update calculate_action.py

* Use assertEntityResponseEqual more

* Update test_capture_new_person for Cloud

* Don't async calculate actions that haven't finished last calculation

* Fix is_calculating condition place

* Improve calculate_actions_from_last_calculation logging

* Clean up event_query, params

* Fix .only usage

* Update migration
2021-03-18 12:30:21 +01:00
Michael Matloka
3c0737f0fd
Plugins access control (#3486)
* Add Organization.PluginsAccess

* Rename PluginsAccess to PluginsAccessLevel

* Use Organization.plugins_access_level in can_…_plugins_via_api

* Add migration for Organization.plugins_access_level

* Remove unused PLUGINS_CLOUD_WHITELISTED_ORG_IDS

* Update access.py

* Add OrganizationPluginsAccessLevel TS enum

* Fix merge

* Disable LocalPlugin UI on Cloud

* Move away from PluginAccess interface

* Extend PluginsAccessLevel range

* Refactor PluginsAccessLevel for brevity

* Remove PluginAccess interface completely

* Add plugins managed globally

* Update migration

* Show managing org name in "Managed" plugin tag

* Smoothen some rough edges

* Smoothen more edges

* Restore correct MULTI_TENANCY default

* All the edges

* Fix most existing tests

* Remove PLUGINS_*_VIA_API env var support

* Update pluginsNeedingUpdates

* Remove can_*_plugins_via_api from instance status page

* Add tests and polish permissioning

* Update migration

* Fix typing

* Make plugin drawer UI less intrusive

* Update migration

* Fix Uninstall button condition

* Use unified _preflight status endpoint instead of the custom plugins one

* Fix plugin update label condition

* Fix "Check for updates" button condition

* Explain PluginsAccessLevel choices with comments

* Hide global plugin installation option on self-hosted

* Don't actions.loadRepository() as install org

* Improve permissioning with tests

* Satisfy mypy

* Add plugins access level to admin and fix org admin

* Check plugins access level more

* Rename endWithPeriod

* Refactor FE access control checks to accessControl.ts

* Deduplicate permissioning

* Add exception message

* Align backend and frontend plugins access level helpers

* Add plugins access level helper tests

* Fix ChartFilter
2021-03-17 15:01:55 +01:00
Tim Glaser
bf2c4429b5
Auto filter test accounts (#3492)
* WIP auto filter test accounts

* finish off

* Fix tests

* Non generic emails

* add list of generic emails

* Move location to below property filters

* Fix typescript errors

* as any

* fix tests

* filters

* fix tests

* Featureflag doesn't really makes sense for this feature

* fix tests

* fix test

* Add clickhouse + tests for each insight

* Fix lifecycle and paths

* Fix sessions

* Fix session tests

* fix sesisons test

* fix migrations

* fix migration chain

* refactor path & remove stale console.log

* adjust useAnchor & minor copy

* rename to avoid confusion with inline component

* test account filter tweaks

* fix filters

* hardcode

* Add tests for funnel trends

* Make generic emails super fast

* Fix migrations

* Default to false for now

* Default to false, remember a user's preference

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-03-11 18:16:38 +01:00
Karl-Aksel Puulmann
5ec765c6a5
Add updated_at to plugins/plugin configs (#3607)
This will be used in plugin server to calculate changes
2021-03-09 16:42:29 +01:00
Paolo D'Amico
78bfade0df
Post-release 1694-dashboards (#3513) 2021-03-03 07:17:43 -08:00
Karl-Aksel Puulmann
f4039003e1
Dashboards: allow changing date filters & refresh (#3363)
* DashboardHeader to ts

* Convert files to typescript

* Use BindLogic instead of passing logics as arguments

* Improve typing for dashboards

* Fix DashboardHeader types

* Improve typing: DashboardItem

* Fix InsightHistoryPanel, SharedDashboard types

* Add WIP code for date ranges on dashboards

* DateFilter/dateFilterLogic to typescript

* Show 'Custom' daterange by default on a dashboard

* Border for dates in Dashboard

* Add dashboard.filters field

* Populate dashboard.filters, use it in code

* Fix some type errors in dashboardItemsModel

* Fetch dashboard items again on date change

* Make dashboard items refresh as dashboard gets updated

* Add refresh button to DashboardHeader

* Make saving dashboard items work again

* Update dashboard type

* Don't update URL in DateFilter

* Update selected time range on load

* Avoid flickering in DateFilter

* Only show 'Custom' in dashboards

* Show calendar icon next to date filter in dashboard

* Avoid double-loading results due to date change

* Kill some duplicated code

* Avoid cache when filters change

(datefilters on dashboard)

* Test individual update_cache method

* Add test for import_from and attributes being updated

* Fix typing errors

* Update api/dashboard tests

* Avoid redirects when changing date range for funnels or retention

Previously funnelLogic might be mounted in the background, causing a
reload

* Add a missing breakpoint

* Reformat with black
2021-02-23 12:19:36 +02:00
Michael Matloka
ce91a06e12
Separate stricter team model migration into two (#3424) 2021-02-22 14:04:25 +01:00
Michael Matloka
65e5878683
Stricter Team model (#3252)
* Remove deprecated Team.opt_out_capture

* Remove obsolete and unused Team.users

* Make Team.organization, api_token and name stricter

* Add migration

* Add Team.api_token min length

* Adjust 0124_stricter_team for opt_out_capture removal

* Don't remove Team fields

* Update migration

* Update migration

* Set org ID of orgless teams to first org instead of deleting

* Update migration

* Fix

* Ensure that each project has a name before migration

* Rename prepare_teams
2021-02-22 13:14:53 +01:00
Karl-Aksel Puulmann
fb3f9d6868
Fix invalid funnel insights display filter (#3309)
* Fix bad default in constants

This was causing invalid funnel visualization, which made the page never
load.

Sentry error: https://sentry.io/organizations/posthog/issues/2153813132/?project=1899813&query=is%3Aunassigned+is%3Aunresolved

* Fix historic data: bad historyitem links
2021-02-11 14:56:59 +02:00
Karl-Aksel Puulmann
a3dca4fa4c
Bump webhook_url limit to 500 chars (#3306)
Sentry error: https://sentry.io/organizations/posthog/issues/2212233625/?project=1899813&query=is%3Aunassigned+is%3Aunresolved

Fixes https://github.com/PostHog/posthog/issues/3304
2021-02-11 12:37:58 +01:00
Tim Glaser
b9f3d3ec7c
Don't set is calculating for static cohorts (#3297)
* Don't set is calculating for static cohorts

* update frontend

* fix inconsistency

* Add migration and fix typing

* N/A

* typing

* types

* more! typing!
2021-02-11 11:07:12 +01:00
Paolo D'Amico
de317c8e87
Bulk invite team members (setup section II) (#3143)
Co-authored-by: Michael Matloka <dev@twixes.com>
2021-02-03 13:03:22 +01:00
Paolo D'Amico
0624471b4a
Setup Section II & General State (#3129) 2021-02-02 17:04:14 +01:00
Tim Glaser
a73651161f
Add person email index (#3116)
* Add team filter to cohort query to speed up

* Fix querying

* Remove emails from CSV upload

* fix clickhouse email

* Add person email index

* back to concurrent
2021-02-02 16:39:05 +01:00
Karl-Aksel Puulmann
ce342cd35f
Onboarding: Save personalization data (#3126) 2021-01-29 13:47:04 +01:00
Michael Matloka
0420786607
Make PluginConfig.order mandatory (#3122)
* Make PluginConfig.order mandatory

* upgrade plugin server to 0.6.10

* black

* Ensure no PluginConfig.order collission

* Clean up PluginConfig API

* Revert "Ensure no PluginConfig.order collission"

This reverts commit d8b2c907e4.

* Black

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-01-28 17:03:36 +01:00
Karl-Aksel Puulmann
249ae881c4
Onboarding: Create demo data, show demo alert at top, route directly to dashboard (#3103)
* Migration for is_demo

* Populate is_demo when triggered

* Refactor: extract create_demo_team

* Also create ee sample data

* Create demo data if feature flag enabled (and in debug), redirect to root

* Show demo warning

* Solve rease issue

* Redirect to personalization

* Redirect to root

* Update antd to latest, which gives access to Alert action prop

* Update demowarning code

* update tests

* Update naming

* Update link
2021-01-27 12:53:10 +02:00
Karl-Aksel Puulmann
1dbbb46f82
Allow configuring session recording retention period (#3074)
* Add session recording retention period column

* Allow setting session recording retention period

Solves #3059

* Run session recording retention task on all self-hosted

Using setting from team.session_recording_retention_period_days
2021-01-26 12:04:39 +02:00
Marius Andra
bfe6f859ec
Plugin upgrades (#3057)
* upgrade typegen

* check for upgrades in the interface

* refactor plugin scene files

* separate plugins needing updates

* show an update button

* show progress when checking

* actually update plugins + ux improvements

* open drawer if new required and unfilled field

* better loading for plugins

* fix frontend type error

* fix backend test

* Do tiny cleanup of version labels

* clarify role of custom.d.ts

* cache plugin update status for 30min, refresh update status on page load if stale

* fix tests

* Fix capitalization in updates check button

* fix bad merge

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-01-26 00:13:47 +01:00