0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-29 03:04:16 +01:00
Commit Graph

134 Commits

Author SHA1 Message Date
Michael Matloka
7fa218bcf8
Add cohort description backend (#6089)
* Add cohort description backend

* Increase `Cohort.description` limit

* Require `DASHBOARD_COLLABORATION` for cohort description box to be shown
2021-09-27 11:57:13 +00:00
Michael Matloka
bc3e223265
Project-based permissioning framework (#5976)
* Refactor `AvailableFeature` from strings to an enum everywhere

* Fix circular dependency and type

* Add "Per-project access" feature flag, premium feature, and organization switch

* Rename `OrganizationMembershipLevel` to `OrganizationAccessLevel`

* Create `ExplicitTeamMembership` model

* Show whether projects are restricted in the project switcher

* Update organizations API code

* Fix migrations

* Move organization tests that require EE to `ee`

* Revert `OrganizationMembershipLevel` rename

* Fix organization tests

* Update migration

* Fix schema and add Members to Project Settings

* Build out test memberships API with security tests

* Update `TeamMembers` and `teamMembersLogic`

* Move "Per-project access" description to tooltip

* Add moar tests

* Fix Project Members list logic

* Add additional membership checks

* Update migrations

* Fix typing

* Adjust explicit team memberships API similarly

* Fix typo

* Unify `ExplicitTeamMemberSerializer`

* Remove old changes to `membersLogic` usage

* Use `effective_membership_level` on `TeamBasicSerializer`

* Clean up organization update tests

* Explicitly disallow enabling per-project access for free

* Fix circular import

* Remove `id` from `UserSerializer`

* Fix typing

* Try to fix import

* Fix fatal typing

* Add more tests

* Update permissioning.ts

* Add clarifying comment to migration

* Fix import

* minor clarifications

* Revert `TopNavigation` changes

* Make new access control entirely project-based

* Update migrations

* Add `project_based_permissioning` to `TeamBasicSerializer`

* Update test_team.py

* Fix Access Control restriction tooltip

* adjust copy & UI a bit

* Address feedback on field comment

* "Privacy settings" to "Access Control"

* Ignore mypy

* Rename `Team` field `project_based_permissioning` to `access_control`

* Update migrations

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-09-22 18:29:59 +02:00
Yakko Majuri
ca57946ff3
add properties_last_updated_at (#6058) 2021-09-22 14:44:14 +00:00
Marius Andra
e13d8e9b6b
Action step empty field ==> null (#5942)
* save null for empty action steps

* save null for empty action steps in toolbar

* simplify

* fix type

* reset existing empty action step fields
2021-09-21 14:04:17 +02:00
Rick Marron
cdd43b908d
Add feature flag overrides api (#5824)
* override feature flags in the API

* improve types (thanks mypy!)

* remove unrelated code

* better comment

* fix test query count

* remove v2 in get_active_feature_flags_v2

* store `$feature/{key}` in properties

* Move feature flag overrides to their own model

* little bit of cleanup

* some type check + test cleanup

* respond to comments, add tests, add team permission checks

* add basic analytics

* add team to override model and simplify endpoints

* cleanup

* fix up instrumentation

* restrict a user to only their own overrides

* override query now works for all distinct ids

* typing fix

* fix tests

* sort flags by created date

* fix tests

* Clean up

* remove defualt from override_value field

* update migration

* fix migration dependancy

* run black on feature_flag.py

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-09-14 09:41:17 -07:00
Yakko Majuri
e708d202c6
Job triggering interface (#5720)
* wip job triggering interface

* add support for number

* update tests

* fix api call

* minor fixes

* move to kea, better UI, remove logs

* updates
2021-09-13 07:01:28 +00:00
Sam Winslow
aedd207a78
Add dive dashboard to model + API (#5779)
* add key for dive dashboard to model

* simplify API & add dive dashboard to it

* walk back sketchy api changes
2021-09-01 22:17:06 +01:00
Karl-Aksel Puulmann
82056d2f05
Add migration to index person by team_id, id DESC (#5489)
Closes https://github.com/PostHog/posthog/issues/4941
2021-08-09 13:57:24 +03:00
Li Yi Yu
cd94321cba
Saved insights default page (#5416) 2021-08-05 07:38:07 -07:00
Michael Matloka
e48bd8b08b
Allow "new member" email opt-out (#5430) 2021-08-04 18:09:41 -07:00
Sam Winslow
60cbf2e7c1
Trigram fuzzy search for property definitions (#5039)
* Trigram fuzzy search for property definitions

* support empty search param

* Only use pg_trgm search if pg extension is installed

* suppress exceptions if migration does not succeed

* checks for index

* troubleshooting

* wip: change migration number; strip out conditionals

* wip: copy code from #4110

* remove is_pg_trgm_installed

* try ngram search on CH

* add AND before ngram

* correct some check failures

* remove ordering

* remove ordering from tests

* fix ordering

* rewrite queries for psql, add test bypass for EE features

* remove ordering filter bc of bugs

* remove test code

* check if isinstance of RawQuerySet

* cleanup sql

* add ordering

* typing

* test for ee prop definition

* adjustments to test_ee_property_definition

* add test_ee_event_definition

* restore this prop

* convert to simple term filter

* use term search in EE

* fix EE tests

* fix non-EE tests

* fix test

* fix filter types

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-07-20 14:58:16 +02:00
Paolo D'Amico
1b190e89cf
SSO domain whitelist (#5111) 2021-07-15 18:20:43 -05:00
Tim Glaser
208d201bba
Remove accidental breakdown on funnels (#5106)
* Remove accidental breakdown on funnels

* fix
2021-07-13 18:17:54 +02:00
Yakko Majuri
6c9bb2db0f
Update token gen (#4832)
* use base36 for random tokens

* Use prefix + base62

* add migration

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-06-23 13:19:05 -03:00
Yakko Majuri
a71a1eac96
add metrics to plugin table (#4783) 2021-06-22 11:07:05 -03:00
Li Yi Yu
7546e3ea0a
Event properties taxonomy (#4486)
* add description and tag fields to event and property definitions

* set up description and tagging on models

* frontend functionality for description editing

* connect backend and kea logic for description editing

* update event and property definitions model and migration

* delete set null instead of cascade

* migration merge fix

* add owner column

* undo posthog event property taxonomy migrations

* set up definitions on enterprise level

* allow enterprise definitions description editing

* fix licensing conditions and add tests

* proper get and update methods for the multi inheritance table and new column fields for enterprise event model

* check for license to separate routes

* migrate existing definitions to ee definitions tables and render ee vs non-ee definition views based on existing feature conditional

* all the working backend updates

* updated tests

* frontend fixes and linting updates

* feature flag it
2021-06-03 09:22:16 -04:00
Paolo D'Amico
1b0240d7fa
Insights short share URL (#4513) 2021-06-01 07:18:19 -07:00
Michael Matloka
eaf2459724
Save Organization.available_features as a DB column (#4426)
* Save Organization.available_features as a DB column

* `update_available_features()` before an organization is created too

* Run the task half past the hour

* Adjust tests and fix available_features sync on start

* Remove redundant null=False

* Fix `Organization.update_available_features()`

* Run Cloud tests in respect to `posthog-cloud` `4426-fix` branch

* Revert "Run Cloud tests in respect to `posthog-cloud` `4426-fix` branch"

This reverts commit 421e8541b3.
2021-05-24 22:17:01 +02:00
Karl-Aksel Puulmann
030529cfb3
Show internal metrics on /instance/status page (#4366)
* Extract function for capture

* Add capability to capture internal metrics

- This is behind an environment variable, we'll turn this on by default
on clickhouse-based installation.
- A special Org/Team gets created for this lazily, which is not accessible for
anyone. This makes avoiding issues with stats/billing/properties and easier.

* Capture some useful metrics into posthog

* Set up an unique constraint for for_internal_metrics organization

This avoids some nasty race conditions

* internal_metrics fixes

* Extract component OverviewTab

* New tabbing system for system status page

* Embed ClickhouseTab

* Generate a dashboard for internal metrics on demand

* More metrics

* Add more internal metrics, add tests

* Fix migration files

* Adjust copy

* Update task definitions

* Fix a typing issue

* Rename a component

* Migration fixup

* Try reset patch after test ends

* Clear lru_cache between tests
2021-05-20 13:42:39 +03:00
Neil Kakkar
7be1b0ef92
Add Capabilities to Plugin model (#4371)
* Add capabilities to plugin model

* update tests
2021-05-18 11:46:01 +01:00
Tim Glaser
1ae890fbe2
1534 configurable columns person events table (#4141)
Co-authored-by: Mohit Gupta <mohitgupta3mail@gmail.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
Co-authored-by: Sam Winslow <sammywinslow@gmail.com>
2021-05-13 14:13:54 -07:00
Michael Matloka
a126e73cb7
Preinstall plugins (#4106)
* Refactor plugin installation and reloading logic

* Preinstall and autoenable plugins (only GeoIP one for now)

* Fix PluginManager.install issues

* Add tests

* Disable plugin preinstallation on Cloud

* Check plugins access level before preinstallation

* Rename field `preinstalled` to `is_preinstalled`

* Adjust tests for Cloud v. self-hosted

* Fix test_preinstalled_are_autoenabled

* Ensure install plugins access level for preinstallation tests

* Bump migration

* Bump migration

* Add and use Plugin.get_default_config, with tests

* Refactor private PluginConfigSerializer methods out of the class

* Skip preinstallation of plugin if it couldn't be done (e.g. airgapped)

* Don't hit GitHub API in tests

* Satisfy mypy

* Fix a test on Cloud CI
2021-05-07 03:02:04 +02:00
Michael Matloka
fb0bd54af6
Increase Element model varchar limits again (#3926)
* Revert "Revert "Increase Element model varchar limits (#3912)" (#3923)"

This reverts commit e30681bade.

* Use RunSQL to avoid ORM crap

* Fix attr_class being converted from varchar[] to varchar

* Bump migration

* Simplify migration and remove attr_class alter due to locking

* Bump migration
2021-05-06 17:11:37 +00:00
Paolo D'Amico
fee0b1035d
Fix lifecycle insights store (#4223)
* lifecycle migration

* update latest migrations

Co-authored-by: eric <eeoneric@gmail.com>
2021-05-06 11:27:42 -04:00
Karl-Aksel Puulmann
9d8b61f2e1
Fix conflicting migrations (#4225)
Error:

```
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0147_fix_stickiness_dashboard_items, 0147_plugin_logs in posthog).
To fix them run 'python manage.py makemigrations --merge'
```
2021-05-06 08:31:57 +00:00
Michael Matloka
e96f95ef5a
Plugin log entries (#3482)
* Add Postgres model PluginLogEntry

* Add equivalent PluginLogEntry to Kafka+ClickHouse

* Add migration

* Add PluginLogEntry.Type.LOG & make PluginLogEntry.message a TextField

* Update 0130_pluginlogentry.py

* Add PluginLogEntry.instance_id

* Update migration

* Update migration

* Add plugin log entries API

* Test plugin log entries DB fetching

* Add PluginLogs component prototype

* Fix API

* Improve PluginLogs component

* Remove almost unused plugin Feedback button

* Update migration

* Fixed typing

* Fix org permission error test asserts

* Fix plugin log entry tests

* Fix CH plugin log entry timestamp string

* Update CH test_plugin_log_entry.py

* Fix plugin log entry tests across PG/CH

* Satisfy mypy

* Add search and limit to plugin log entry API

* Send team_id in plugin config API

* Rework plugin logs UI

* Add plugin config team ID in tests

* Add plugin config team ID in tests actually

* Fix code quality

* Make logs plugin config-based

* Fix CH queries

* Fix typing

* Improve UX and fix things

* Polish plugin logs logic

* Update migration

* Add Celery task to delete old plugin logs

* Fix UX bug with loading more plugin logs

* Fix missing import

* Remove OrganizationMemberPermissions message change

* Make mypy happy

* Add PluginLogEntry.is_system

* Optimize CH plugin_log_entires PARTITION/ORDER

* Increment migration

* Adjust plugin logs drawer display

* Fix plugin_log_factory_ch

* Fix plugin_log_factory_ch fix

* Replace PluginLogEntry.is_system with source

* Adjust PluginLogEntrySerializer

* Update CH fetch_plugin_log_entries

* Make kea-typegen happy
2021-05-06 10:54:32 +03:00
Marius Andra
ef1a5470a9
Migration to sync team event properties (#4101)
* migration to add delayed jobs to sync team event properties

* remove .delay

* update latest migrations

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-04-26 13:16:48 +03:00
Paolo D'Amico
072a6c1beb
Refactor event & properties for async server-side loading (#4070)
* remove event & properties endpoint from team endpoint

* event definition model

* sync events from Team with EventDefinition

* fix typing

* missing commit

* event definition API

* add querying capabilities

* property definition model

* sync property definitions

* temp: bring back event properties to Team model to split PR

* PropertyDefinition API + tests

* fix tests for Python 3.7 & more deprecation notices

* default values to 0 instead of null

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-04-22 08:42:01 +00:00
Paolo D'Amico
7ea9eb692c
Update Django to 3.1.8 (and related) (#4007) 2021-04-19 12:59:10 -07:00
Tim Glaser
5cfbc49cc3
Add clickhouse and max users to license (#3918)
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-04-14 08:45:39 -07:00
Michael Matloka
e30681bade
Revert "Increase Element model varchar limits (#3912)" (#3923) 2021-04-08 21:24:03 +02:00
Michael Matloka
305e5f7210
Increase Element model varchar limits (#3912) 2021-04-08 20:17:29 +02:00
Paolo D'Amico
9865bbc3c1
User V2 API endpoint (#3855)
* initial refactor

* simple serializer for teams

* basic team in front-end

* use basicTeam where applicable

* refactor UserSerializer to UserBasicSerializer

* abstract UserBasicSerializer

* add uuid to users

* move event properties methods to Team model

* basic V2 user endpoint

* update current user

* update current team and org

* post-merge fix

* fix load logic

* change password

* add some extra tests

* undo all frontend changes from this PR

* remove last frontend change

* update preflight tests

* remove team basic fetch

* fix foss tests

* adjust migration & add distinct ID to payload

* add deprecation notice

* add uuid to UserBasic serializer & update memberships

* add available features to org serializer

* address feedback

* endpoint -> `users/@me/`

* address feedback

* Use UUIDClassicModel abstract model when adding uuid field

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-04-05 21:42:31 +02:00
Michael Matloka
2c8b63b68a
Fix Team.data_attributes default (#3846)
* Fix Team.data_attributes default

* Add test
2021-04-01 21:06:45 +02:00
Marius Andra
581a617a95
Add index to events.created_at (#3767)
* add index to events.created_at

* convert to concurrent index
2021-03-30 10:43:24 +02:00
Marius Andra
7ed7c4c649
Custom Data Attributes in the toolbar (#3761)
* add `important_data_attributes`

* fix undefined bug

* important data attributes in setup

* add dashes

* remove dead code

* scarier exampe

* support multiple data attributes in the toolbar

* update to simmer that supports wildcards

* rename "important_data_attributes" -> "data_attributes", store as a json array

* update ts blacklist

* another one

* use a tag select for data attributes

* remove needless string serialization

* implement text feedback

* data attribute setup style improvement

* better default handling

* fix out of sync migration
2021-03-30 10:28:57 +02:00
Paolo D'Amico
da61c55666
Description & tagging on dashboards (#3756) 2021-03-26 17:38:13 -07:00
Paolo D'Amico
136a021d1a
Feature Flags Revamp (UX & Code) (#3675) 2021-03-23 14:34:48 -07:00
Paolo D'Amico
a8838701aa
Timezone handling UX (#3698)
Co-authored-by: Tim Glaser <tim@glsr.nl>
Co-authored-by: Michael Matloka <dev@twixes.com>
2021-03-23 12:54:33 -07:00
Michael Matloka
ac5fa0686b
Allow for non-plugin config-keyed plugin attachments (#3732) 2021-03-23 18:46:56 +01:00
Michael Matloka
828e0fd5c2
Solidify plugins on Cloud rollout (#3706)
* Better handle extra info buttons during plugin reordering

* Fix UI bug in first time plugin enabling

* Give very new Cloud org config access to plugins

* Add migration
2021-03-19 14:25:09 +01:00
Michael Matloka
3c571f1312
Different async action mapping approach (#3678)
* Disable useless InsecureRequestWarning

* Disallow ASYNC_EVENT_ACTION_MAPPING on CH and make it default otherwise

* Return early from calculate_actions_from_last_calculation if CH

* Optimize Action.calculate_events and fire hooks from it

* Run calculate_event_action_mappings every 30 s instead of 5 min

* Optimize post_event_to_webhook thanks to new actions approach

* Add Event.site_url field for reworked Postgres webhooks

* Adjust tests by using sync calls to actions calculation

* Update calculate_action.py

* Use assertEntityResponseEqual more

* Update test_capture_new_person for Cloud

* Don't async calculate actions that haven't finished last calculation

* Fix is_calculating condition place

* Improve calculate_actions_from_last_calculation logging

* Clean up event_query, params

* Fix .only usage

* Update migration
2021-03-18 12:30:21 +01:00
Michael Matloka
3c0737f0fd
Plugins access control (#3486)
* Add Organization.PluginsAccess

* Rename PluginsAccess to PluginsAccessLevel

* Use Organization.plugins_access_level in can_…_plugins_via_api

* Add migration for Organization.plugins_access_level

* Remove unused PLUGINS_CLOUD_WHITELISTED_ORG_IDS

* Update access.py

* Add OrganizationPluginsAccessLevel TS enum

* Fix merge

* Disable LocalPlugin UI on Cloud

* Move away from PluginAccess interface

* Extend PluginsAccessLevel range

* Refactor PluginsAccessLevel for brevity

* Remove PluginAccess interface completely

* Add plugins managed globally

* Update migration

* Show managing org name in "Managed" plugin tag

* Smoothen some rough edges

* Smoothen more edges

* Restore correct MULTI_TENANCY default

* All the edges

* Fix most existing tests

* Remove PLUGINS_*_VIA_API env var support

* Update pluginsNeedingUpdates

* Remove can_*_plugins_via_api from instance status page

* Add tests and polish permissioning

* Update migration

* Fix typing

* Make plugin drawer UI less intrusive

* Update migration

* Fix Uninstall button condition

* Use unified _preflight status endpoint instead of the custom plugins one

* Fix plugin update label condition

* Fix "Check for updates" button condition

* Explain PluginsAccessLevel choices with comments

* Hide global plugin installation option on self-hosted

* Don't actions.loadRepository() as install org

* Improve permissioning with tests

* Satisfy mypy

* Add plugins access level to admin and fix org admin

* Check plugins access level more

* Rename endWithPeriod

* Refactor FE access control checks to accessControl.ts

* Deduplicate permissioning

* Add exception message

* Align backend and frontend plugins access level helpers

* Add plugins access level helper tests

* Fix ChartFilter
2021-03-17 15:01:55 +01:00
Tim Glaser
bf2c4429b5
Auto filter test accounts (#3492)
* WIP auto filter test accounts

* finish off

* Fix tests

* Non generic emails

* add list of generic emails

* Move location to below property filters

* Fix typescript errors

* as any

* fix tests

* filters

* fix tests

* Featureflag doesn't really makes sense for this feature

* fix tests

* fix test

* Add clickhouse + tests for each insight

* Fix lifecycle and paths

* Fix sessions

* Fix session tests

* fix sesisons test

* fix migrations

* fix migration chain

* refactor path & remove stale console.log

* adjust useAnchor & minor copy

* rename to avoid confusion with inline component

* test account filter tweaks

* fix filters

* hardcode

* Add tests for funnel trends

* Make generic emails super fast

* Fix migrations

* Default to false for now

* Default to false, remember a user's preference

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-03-11 18:16:38 +01:00
Karl-Aksel Puulmann
5ec765c6a5
Add updated_at to plugins/plugin configs (#3607)
This will be used in plugin server to calculate changes
2021-03-09 16:42:29 +01:00
Paolo D'Amico
78bfade0df
Post-release 1694-dashboards (#3513) 2021-03-03 07:17:43 -08:00
Karl-Aksel Puulmann
f4039003e1
Dashboards: allow changing date filters & refresh (#3363)
* DashboardHeader to ts

* Convert files to typescript

* Use BindLogic instead of passing logics as arguments

* Improve typing for dashboards

* Fix DashboardHeader types

* Improve typing: DashboardItem

* Fix InsightHistoryPanel, SharedDashboard types

* Add WIP code for date ranges on dashboards

* DateFilter/dateFilterLogic to typescript

* Show 'Custom' daterange by default on a dashboard

* Border for dates in Dashboard

* Add dashboard.filters field

* Populate dashboard.filters, use it in code

* Fix some type errors in dashboardItemsModel

* Fetch dashboard items again on date change

* Make dashboard items refresh as dashboard gets updated

* Add refresh button to DashboardHeader

* Make saving dashboard items work again

* Update dashboard type

* Don't update URL in DateFilter

* Update selected time range on load

* Avoid flickering in DateFilter

* Only show 'Custom' in dashboards

* Show calendar icon next to date filter in dashboard

* Avoid double-loading results due to date change

* Kill some duplicated code

* Avoid cache when filters change

(datefilters on dashboard)

* Test individual update_cache method

* Add test for import_from and attributes being updated

* Fix typing errors

* Update api/dashboard tests

* Avoid redirects when changing date range for funnels or retention

Previously funnelLogic might be mounted in the background, causing a
reload

* Add a missing breakpoint

* Reformat with black
2021-02-23 12:19:36 +02:00
Michael Matloka
ce91a06e12
Separate stricter team model migration into two (#3424) 2021-02-22 14:04:25 +01:00
Michael Matloka
65e5878683
Stricter Team model (#3252)
* Remove deprecated Team.opt_out_capture

* Remove obsolete and unused Team.users

* Make Team.organization, api_token and name stricter

* Add migration

* Add Team.api_token min length

* Adjust 0124_stricter_team for opt_out_capture removal

* Don't remove Team fields

* Update migration

* Update migration

* Set org ID of orgless teams to first org instead of deleting

* Update migration

* Fix

* Ensure that each project has a name before migration

* Rename prepare_teams
2021-02-22 13:14:53 +01:00
Karl-Aksel Puulmann
fb3f9d6868
Fix invalid funnel insights display filter (#3309)
* Fix bad default in constants

This was causing invalid funnel visualization, which made the page never
load.

Sentry error: https://sentry.io/organizations/posthog/issues/2153813132/?project=1899813&query=is%3Aunassigned+is%3Aunresolved

* Fix historic data: bad historyitem links
2021-02-11 14:56:59 +02:00