0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-25 11:17:50 +01:00
Commit Graph

42 Commits

Author SHA1 Message Date
Neil Kakkar
6017a39d7e
Install scipy (#7897)
* install scipy
* pin
* sort out versions
2022-01-05 13:06:37 +00:00
Yakko Majuri
322e253413
Add version requirements for deploying (#7283)
* add version requirements for deploying

* update postgres range

* postgres -> postgresql, version_range -> supported_version

* updates

* add test, revert ee app change

* fix

* fix

* rename to service version requirement

* add redis

* remove ee requirements from ee settings

* update redis v

* update postgres v

* Update posthog/version_requirement.py

Co-authored-by: Guido Iaquinti <4038041+guidoiaquinti@users.noreply.github.com>

* add SKIP_SERVICE_VERSION_REQUIREMENTS setting

* add warning for skipping reqs

* update redis version

* skip service req checks for collecstatic

* update settings

Co-authored-by: Guido Iaquinti <4038041+guidoiaquinti@users.noreply.github.com>
2021-11-23 12:44:55 +00:00
Guido Iaquinti
b4a6ece4ec
Dockerfile(s) - use 'libpq~=13' (#7113) 2021-11-15 11:00:08 +02:00
Yakko Majuri
963c62aaf1
Use monorepo plugin server (#6784)
* use plugin-server in same repo

* remove plugins dir

* update docker files

* clean up docker files:

* visual separation

* run plugin server test stack

* build and push test image

* revert push image

* fix default yarn start

* another fix

* fix some more

* add plugin-server-prod script

* make executable

* update scripts:

* revert prod script

* fixes

* fix tests

* clean

* move build to docker

* build plugin-server in prod dockerfile

* use different entrypoint for e2e test

* fix bugs with ts-node-dev

* restore yarn.lock

* lost a few changes in the rebase

* fix dockerfile

Co-authored-by: James Greenhill <fuziontech@gmail.com>
2021-11-13 11:52:00 -08:00
Guido Iaquinti
ff07577a47
Move production.Dockerfile from slim to alpine (#6575)
* Move production.Dockerfile from slim to alpine

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Update production.Dockerfile

Co-authored-by: Harry Waye <harry@scalexp.com>

* WIP

* Don't remove 'plugins/node_modules'

Co-authored-by: Harry Waye <harry@scalexp.com>
2021-11-11 10:42:59 -08:00
Guido Iaquinti
8d21741584
Add 'libpq-dev' to production.Dockerfile (#6610) 2021-10-22 14:46:49 +02:00
Guido Iaquinti
0627032b1c
Split Dockerfile lint to dedicated GH action (#6468) 2021-10-18 12:51:46 +00:00
Paolo D'Amico
89b0a1335b
Optimize Dockerfile build (#5988) 2021-09-17 09:22:12 +02:00
Paolo D'Amico
cf9814c08f
SAML deps build (#5932) 2021-09-14 11:43:26 +02:00
Paolo D'Amico
ef7ec14014
Conditional SAML dependencies (fix Heroku deploys) (#5870) 2021-09-09 14:09:15 +00:00
James Greenhill
9a65de76c6
Attempt a fix for building xmlsec python wheel (#5768) 2021-08-28 13:47:25 +02:00
Paolo D'Amico
627d493817
SAML support (#5681) 2021-08-27 07:20:49 -07:00
Karl-Aksel Puulmann
8acb0e2242
Fix docker build (#5609) 2021-08-17 10:55:40 +03:00
Paolo D'Amico
7ea9eb692c
Update Django to 3.1.8 (and related) (#4007) 2021-04-19 12:59:10 -07:00
Adrien Brault
864dd60a18
Make docker apt version pinning less strict (#3889)
* Make docker apt version pinning less strict

* Single quote versioned apt-get install packages

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-04-07 12:51:34 +02:00
Adrien Brault
e930e4c243
Fix apt-get install error when building docker images (#3833)
Thanks for fixing this @adrienbrault!
2021-03-31 17:53:40 -07:00
Paolo D'Amico
26ad3808d7
Dockerfile non-root user in production (#3735)
Co-authored-by: jake emo <jre200@vt.edu>
Co-authored-by: Michael Matloka <dev@twixes.com>
2021-03-23 21:32:01 -07:00
Michael Matloka
9f24da6741
Lint Dockerfiles (#3663)
* Lint with Hadolint in Docker test workflow

* Try fixing Hadolint install in workflow

* Try /bin/ instead of /usr/bin/

* Install Hadolint differently

* Install Hadolint differently differently

* Fix Hadolint install properly

* Use chmod +x

* Download Hadolint to workdir

* Update docker-image-test.yml

* Silence curl

* Address production.Dockerfile linter warnings

* Delete obsolete preview.Dockerfile

https://posthog.slack.com/archives/C0113360FFV/p1614948302111800?thread_ts=1614946698.107000&cid=C0113360FFV

* Address dev.Dockerfile linter warning

* Clean up production.Dockerfile
2021-03-18 18:03:42 +01:00
Marius Andra
24a6ab89c9
re-enable source maps in docker (#2879) 2021-01-07 16:50:07 +01:00
James Greenhill
5419370f79
Clean up prod dockerfile, remove dupe calls (#2806) 2020-12-16 20:48:15 -08:00
Marius Andra
c191fef86b
Ignore plugins optional deps (#2801)
* ignore optional deps when installing plugin-server

* --ignore-optional in docker
2020-12-16 23:59:40 +01:00
Marius Andra
29824fbfed
Fix Docker Build (#2797)
* add build-essential

* also purge it
2020-12-16 17:28:30 +01:00
Michael Matloka
259e801de6
Whitelist plugins per organization on Cloud (#2791)
* Fix unix-dgram build

* Add per-organization whitelisting of plugins API on Cloud

* Add migration

* Update can_install_plugins_via_api calls in serializer

* Remove unused type: ignore

* Update PluginSerializer for organization_id

* Make Plugin.organization nullable

* Fix can_install_plugins_via_api

* Update pluginsLogic.ts

* Adjust can_configure_plugins_via_api for organization

* Don't include organization_id in serializer

* Set default Plugin.organization to first Organization in instance

* Fix PluginConfigSerializer.create

* Test that other orgs' plugins can't be accessed (as if they didn't exist, 404)

* fix typo

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2020-12-16 14:54:30 +01:00
Marius Andra
eb8b650aee
Docker nodejs support for plugins (#2452)
* keep npm around in the dockerfiles (needed for plugin-server)

* simplify docker-preview script
2020-11-20 17:04:10 +01:00
Michael Matloka
b7b9e8caf3
Remove some seemingly unused requirements and use pip-compile (#2192)
* Remove some seemingly unused requirements

* Remove a few more

* Bring back dependencies of other requirements

* pip-compile prod requirements

* Sync with master updates

* Fix grep

* Revert "Fix grep"

This reverts commit 2e6538cad4.

* Fix grep but using cut

* Clean up and regenerate requirements

* Simplify requirements-dev

* Fix fakeredis import in production

* Black
2020-11-10 10:17:48 +01:00
Marius Andra
fce86ef46d
Plugin v8 (#1946)
* plugin progress

* blah

* add posthog config for plugins

* test gitignore

* new functionality for plugins

* support local plugin paths

* also ignore symlinks

* add positional argument

* fixes

* small fixes

* config polish

* config passed to posthog plugin

* ooooooops

* symlink fix

* cleanse dir before loading

* add cache to plugins

* pickle the goods

* unlink symlink

* pass full config

* unlink even if link points to nothing

* fix fix

* return none if value is empty

* plugin model

* plugins scene

* add config schema to plugins

* install plugins

* save descriptions

* show descriptions

* edit plugin

* save plugin config

* plugin modal

* uninstall plugins

* UX cleanup

* add "required" to plugin config

* open plugin modal after install

* split to subcomponents

* install custom plugins

* rework backend for model plugins

* Plugins on models

* simple reload pubsub

* fix apps not installed

* fix master/main issue

* fix reload command

* use the github api to get the default branch

* init plugins only if not running migrate/makemigrations

* store plugins zip archives in postgres

* tag plugins to specific versions

* save plugins in pluginConfig

* update pluginConfigs instead of adding new rows, remove from redux on uninstall

* remove debug

* run plugins from db by team

* reload when deleting

* remove debug

* smarter handling of dynamic plugins, support local plugins again

* improve typings, add some nicer warnings

* yarn lock file after merge

* squash migrations and add "locked" field to plugins

* error if folder not found in zip

* unregister plugins

* skip plugin init in test mode

* basic plugin test

* avoid mutating the same prop hash

* add pip tools to requirements.txt

* fix mypy, fix manage.py script error

* avoid plugins with mypy

* mypy fix

* abstract redis into plugin and add team_id to reload

* refactor and start work on syncing with posthog.json

* start testing plugin loading from json

* test plugin deletion

* test for syncing plugins from config

* complete and then test local json plugin sync

* test converting back and forward between an local and http path

* remove global plugin config from plugins array in posthog.json

* rename configSchema --> config_schema

* fix migration after merge

* rename from_cli to from_json

* mypy

* import pip after plugin loaded

* show error details

* raise exceptions visible to the frontend

* sync plugins on load

* access control to updating plugins from the web

* access control

* remove posthog.json from git

* test config schema from json

* if you can install via the web, you can also configure

* remove separate view access

* title as "Plugins" instead of "Installed Plugins" if we can't install ourselves and don't see the repository

* add self.team to plugin base class

* add instance_init method

* refactor into files

* sync global plugin config from json

* make global plugins work, add test

* global plugins in interface, make them take precedence over local plugins

* add comments to plugin base class

* reload/reset plugins before each test

* add error field to plugins

* add many plugin zips

* add many plugin zips, fix imports

* store errors on plugin object and test them

* fix types

* add null to error

* can be with any team ID in the test

* save problems running plugins in the plugin_config model

* try to create redis connection pool only once

* throw if no redis

* mypy

* get instance inside heartbeat and not top level

* try caching pubsub

* try pip install with -q

* install pip externally

* remove uuid and typing, now in stdlib

* more verbosity

* add pip back

* catch exceptions

* quiet and no input for pip

* check plugin reload every 10sec on new task

* fix type errors

* fix requirements error message

* use repository.json

* only load and reload plugins on workers

* rename task

* support local js plugins via py-mini-racer

* load js plugins from zip files

* extract jsplugin class and convert to syntax that uses global functions instead of initializing a class

* process events via grpc

* process events with the "posthog-plugins" queue to enable plugins

* remove old native python & mini racer plugin code

* default to false

* change env vars

* fix test

* remove grpc tools

* skip plugins in migrate.py scripts

* fix migration

* change output of settings debug banner to STDERR

* start posthog plugin server with worker

* try to fix python 3.7 test

* add fallback for the optional argument

* annoying CI test debug

* try to finally fix python 3.7 test

* here we go again

* move plugins under instance

* move plugins npm start into its own folder

* more console.log debugginf

* and again

* move plugins to separate script

* more prints

* fix test error

* docker config

* small fixes

* dckerfile fix

* reload plugins via pubsub, upgrade version

* plugins that support team setup code

* sync if made changes from config

* move plugins in menu

* require node 14 in heroku for better plugin support (namely ?. support)

* bump node version in dockerfiles

* update node versions for github actions

* update the concurrency for heroku workers

* update the concurrency for heroku workers (add link)

* Fix migrations after merge

* add ignore_result to process_event tasks

* fix: docker-preview run in parallel bug

* change order of commands

* remove separate plugins server conf script

* clarify intent

* revert castaway change

* add context to plugins/sync.py

* change everything to ValidationError

* delegate destroy to super

* no request to repository url if can't install

* make the if cleaner

* add clarifying line

* add clarifying line

* fix url field type

* rename get_redis_instance to get_client, move to posthog.redis

* remove duplicate validation

* flip if around

* simplify api logic

* simplify plugin_config api, fix global_plugin error

* remove unnecessary field rename

* mypy

* Plugins UI (#2090)

* base UI

* more UI

* load plugin image if available

* toggle enabled plugin

* plugin cards for available

* custom plugins

* change plugin configuration to drawer

* asks for confirmation when enabling or disabling a plugin

* loading state

* separation of concerns, leave new styles for separate PR

* general improvements

* remove button when installation is not available

* preemptively avoid merge conflict with #2114

* move papercups widget & hide bottom bar when drawer is open

* allow clicking the entire plugin card

* address all feedback

* move plugins under "project" menu

* Hide "configure" from globally enabled plugins

Co-authored-by: Marius Andra <marius.andra@gmail.com>

* add plugins opt in toggle to project/plugins

* choose pipeline based on team setting

* add "beta"

* plugin opt-in opt-out pages

* adjust install button

* remove tasks that are never called, remove PLUGINS_ENABLED global key

* fix responsive card display

* fix typo and drawer width

* skeleton fixes

* typo

* use "posthog-plugin-server" npm package

* "posthog-plugin-server" doc

* require the plugin server to be online before enabling plugins

* remove a few needeless "?." cases

* add hint for config_schema

* add hint for errors

* show plugin errors

* stop clicks if clicking on error

* show plugin errors

* loading indicators

* reload plugins when opting in/out

* nicer beta tag

* add frontend type

* fix mypy error

* fix test

* disable plugins if MULTI_TENANCY

* upgrade plugin-server version

* save event with plugin error

* upgrade plugin server

* squashed & optimized migrations

* remove unused import

* updates opt-in copy & hides tech details for cloud version

* fix cypress tests

* compare with None

* change plugins url and add redirect

* remove ellipsis

* use code snippet in plugin errors

* change github regex

* fix loading flickering on installing plugins

* add comment to plugin archive

* fix python style

* remove pip-tools (relic from the python plugin era)

* hard pin plugin server version

* remove copying of posthog.json from dev dockerfile (breaks if file doesn't exist, copied later anyway)

* update lockfile

Co-authored-by: James Greenhill <fuziontech@gmail.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-11-02 15:08:30 +01:00
Tim Glaser
b3df75bde5
Add docker image test (#1515)
* Add docker image test

* Fix test?

* Fix test? bis

* Document build fix

Co-authored-by: Michael Matloka <dev@twixes.com>
2020-09-01 16:02:53 +02:00
Michael Matloka
c040601f49
Personal API keys and Zapier integration (#1281)
* Add missing migration

* Add generate_random_token() model util

* Move PublicTokenAuthentication to utils

* Make use of generate_random_token

* Add User.personal_access_token field

* Add PersonalAccessTokenAuthentication

* Fix PublicTokenAuthentication

* Fix migration and auth import

* Add personal_access_token to user API

* Update Setup.js

* Support trailing slash in API

* Improve PAT auth quality

* Add django-rest-hooks requirement

* Update settings.py for rest_hooks

* Fix django-rest-hooks requirement

* Bring back API routes with no double trailing slash

* Rename posthog.api.team to team_user

* Add API TODO

* Ad PAT auth with X-PAT HTTP header

* Replace User.personal_access_token with PersonalAPIKey model

* Fix PersonalAPIKey max_lengths

* Describe posthog.models.utils.generate_random_token better

* Add personal_api_key to API

* Add authenticate_header to PersonalAPIKeyAuthentication

* Add hook API endpoint

* Use django.utils.timezone in place of datetime.datetime

* Add Personal API Keys to Setup

* Sort personal_api_keys in ORM

* Add Action.on_perform()

* Remove requirements.txt comment

* Add a

* Add REST hook tasks

* Optimize PersonalAPIKeyAuthentication query

* Add a trailing slash version of /e endpoint

* Add team field to PersonalAPIKey model

* Add personal API key support to capture endpoint, get_cached_from_token

* Reject personal API keys from inactive users

* Add extra_properties_json field to /capture

* Improve PAK auth header regex

* Use custom hook model

* Deliver hooks

* Handle action.on_perform

* Consolidate userLogic in userLogic.tsx

* Update PersonalAPIKeys.js

* Make PersonalAPIKey foreign keys read-only

* Update requirements/dev.txt

* Make PersonalAPIKeys TSX

* Fix conflict

* Fix migration

* Fix minor mishaps

* Update and fix tests

* Use CharField of random 32 bits as hook.id

* Fix conflicting migrations

* Fix ValidationError in HookSerializer.validate_event

* Use query param in /api/event/actions ID filtering

* Rename endpoint `hook` to `hooks`

* Satisfy mypy

* Add tests

* Use DRF serialization in action_defined and annotation_created triggers

* Update migration leafs

* Make mypy ignore rest_hooks

* Update Django signal receiver names

* Update TS dependencies

* Revert "Update TS dependencies"

This reverts commit 7fc26fefcd.

* Add field user to Hook model

* Update migration leafs

* Fix circular import

* Fix some code

* Install git before running pip install in Dockerfiles

* Improve personal API keys UI

* Satisfy mypy

* Reword key label placeholder

* Add personal API key support to /api/user/*

Unfortunately these endpoints are still limited by CSRF protections at the moment, so not accessible outside PostHog itself.

* Improve PersonalAPIKeyAuthentication and add CsrfOrKeyViewMiddleware

* Run collectstatic before test

* Don't install dev dependencies in CI

* Update dependency installation order in CI

* Fix bug and describe PersonalAPIKeyAuthentication

* Fix CI issues

* Fix typing issues

* Fix more typing issues

* Use /api/personal_api_keys to list keys

* Move REST hooks (and therefore Zapier) to ee/

* Refactor personal API logic with kea-loaders

* Add "More about API authentication in PostHog docs."

* Update PersonalAPIKeys.tsx

* Use TestMixin

* Fix "Authentication" that should've been "Authorization"

* Add option to skip self.client.force_login in API tests

* Include team_id and user_id in personal API key serialization

* Update test_hooks.py

* Add personal API key tests

* Remove leftover

* Make ee.settings override posthog.settings

* Don't directly import from models

* Remove unused imports

* Fix mypy issues

* Fix HOOK_DELIVERER

* Use decorator for /api/user PAK auth

* Don't fire REST hook if user doesn't have "zapier" feature

* Import Optional

* Reword to "premium Zapier"

* Make mypy happy

* Fix test_delete_personal_api_key

* Fix misclick

* Fix and test /capture with personal API key

* Make mypy happy

* Remove extra_properties_json

* Resolve migrations

* Remove apt-utils

* Optimize and test PAK user.is_active filtering

* Replace DEBUG true with 1

* Remove unused instance_id

* Improve typing

* Fix deletion toast

* Refactor CopyToClipboard and use it in PAKs

* Use toast.success

* Update migrations

* Fix migration

* Fix migrations

* Complete merge

Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-08-26 10:34:57 +02:00
Marius Andra
f1c90f4283
Whitelist kea-typegen folder in .dockerignore (#1443)
* Whitelist kea-typegen folder in .dockerignore

* Added debug to devdockerfile

* quote style fix

* add .kearc to dockerignore (needed to generate types in yarn build)

* Add tsconfig.json to dockerfile

* add tsconfig and kearc to dockerfiles

* updated docker files to suppress secret key error

* use fake secret key in collectstatic

Co-authored-by: Yakko Majuri <yakko.majuri@gmail.com>
2020-08-17 14:11:46 +02:00
Tim Glaser
302d587d78
Copy correct babel file in docker images (#1194) 2020-07-13 11:20:21 +02:00
Tim Glaser
dfd34a6726
Remove dev requirements (#826) 2020-05-26 14:38:33 +02:00
Sanket Dasgupta
3738f33e30
Remove drf-yasg in favor of new docs website (#596)
The new documentation at https://docs.posthog.com supersedes
this.

Fixes https://github.com/PostHog/posthog/issues/574
2020-04-15 16:24:10 +01:00
Tim Glaser
52a6b621e6
Fix network timeout yarn for antd (#551) 2020-04-13 10:53:37 +01:00
Marius Andra
8798ad8d4e celery/redis workers in preview.Dockerfile 2020-04-07 14:01:33 +02:00
Tim Glaser
ccdfe26a40 Remove yarn cache in production image 2020-04-07 11:05:15 +01:00
Marius Andra
6a36c304f6 collect static inside docker 2020-04-02 00:55:29 +02:00
Marius Andra
dcf565f39d add yarn to dockerfiles 2020-03-18 13:41:46 +01:00
Marius Andra
3a943e1c11 install in docker via root package.json and webpack 2020-03-18 13:41:46 +01:00
Tim Glaser
698e7bbc47 Closes #201 docker entrypoint to CMD 2020-02-26 14:02:06 -08:00
Tim Glaser
4e59812b6b Add API documentation 2020-02-13 22:50:44 -08:00
Tim Glaser
d55d7f86e2 Fix psycopg2 dependency issues by having Heroku use psycopg2 and docker psycopg2-binary 2020-02-11 19:54:58 -08:00
Tim Glaser
57e40b5c59 Create preview and production dockerfiles and add LICENSE 2020-02-09 13:54:14 -08:00