Plugin v8 (#1946)
* plugin progress
* blah
* add posthog config for plugins
* test gitignore
* new functionality for plugins
* support local plugin paths
* also ignore symlinks
* add positional argument
* fixes
* small fixes
* config polish
* config passed to posthog plugin
* ooooooops
* symlink fix
* cleanse dir before loading
* add cache to plugins
* pickle the goods
* unlink symlink
* pass full config
* unlink even if link points to nothing
* fix fix
* return none if value is empty
* plugin model
* plugins scene
* add config schema to plugins
* install plugins
* save descriptions
* show descriptions
* edit plugin
* save plugin config
* plugin modal
* uninstall plugins
* UX cleanup
* add "required" to plugin config
* open plugin modal after install
* split to subcomponents
* install custom plugins
* rework backend for model plugins
* Plugins on models
* simple reload pubsub
* fix apps not installed
* fix master/main issue
* fix reload command
* use the github api to get the default branch
* init plugins only if not running migrate/makemigrations
* store plugins zip archives in postgres
* tag plugins to specific versions
* save plugins in pluginConfig
* update pluginConfigs instead of adding new rows, remove from redux on uninstall
* remove debug
* run plugins from db by team
* reload when deleting
* remove debug
* smarter handling of dynamic plugins, support local plugins again
* improve typings, add some nicer warnings
* yarn lock file after merge
* squash migrations and add "locked" field to plugins
* error if folder not found in zip
* unregister plugins
* skip plugin init in test mode
* basic plugin test
* avoid mutating the same prop hash
* add pip tools to requirements.txt
* fix mypy, fix manage.py script error
* avoid plugins with mypy
* mypy fix
* abstract redis into plugin and add team_id to reload
* refactor and start work on syncing with posthog.json
* start testing plugin loading from json
* test plugin deletion
* test for syncing plugins from config
* complete and then test local json plugin sync
* test converting back and forward between an local and http path
* remove global plugin config from plugins array in posthog.json
* rename configSchema --> config_schema
* fix migration after merge
* rename from_cli to from_json
* mypy
* import pip after plugin loaded
* show error details
* raise exceptions visible to the frontend
* sync plugins on load
* access control to updating plugins from the web
* access control
* remove posthog.json from git
* test config schema from json
* if you can install via the web, you can also configure
* remove separate view access
* title as "Plugins" instead of "Installed Plugins" if we can't install ourselves and don't see the repository
* add self.team to plugin base class
* add instance_init method
* refactor into files
* sync global plugin config from json
* make global plugins work, add test
* global plugins in interface, make them take precedence over local plugins
* add comments to plugin base class
* reload/reset plugins before each test
* add error field to plugins
* add many plugin zips
* add many plugin zips, fix imports
* store errors on plugin object and test them
* fix types
* add null to error
* can be with any team ID in the test
* save problems running plugins in the plugin_config model
* try to create redis connection pool only once
* throw if no redis
* mypy
* get instance inside heartbeat and not top level
* try caching pubsub
* try pip install with -q
* install pip externally
* remove uuid and typing, now in stdlib
* more verbosity
* add pip back
* catch exceptions
* quiet and no input for pip
* check plugin reload every 10sec on new task
* fix type errors
* fix requirements error message
* use repository.json
* only load and reload plugins on workers
* rename task
* support local js plugins via py-mini-racer
* load js plugins from zip files
* extract jsplugin class and convert to syntax that uses global functions instead of initializing a class
* process events via grpc
* process events with the "posthog-plugins" queue to enable plugins
* remove old native python & mini racer plugin code
* default to false
* change env vars
* fix test
* remove grpc tools
* skip plugins in migrate.py scripts
* fix migration
* change output of settings debug banner to STDERR
* start posthog plugin server with worker
* try to fix python 3.7 test
* add fallback for the optional argument
* annoying CI test debug
* try to finally fix python 3.7 test
* here we go again
* move plugins under instance
* move plugins npm start into its own folder
* more console.log debugginf
* and again
* move plugins to separate script
* more prints
* fix test error
* docker config
* small fixes
* dckerfile fix
* reload plugins via pubsub, upgrade version
* plugins that support team setup code
* sync if made changes from config
* move plugins in menu
* require node 14 in heroku for better plugin support (namely ?. support)
* bump node version in dockerfiles
* update node versions for github actions
* update the concurrency for heroku workers
* update the concurrency for heroku workers (add link)
* Fix migrations after merge
* add ignore_result to process_event tasks
* fix: docker-preview run in parallel bug
* change order of commands
* remove separate plugins server conf script
* clarify intent
* revert castaway change
* add context to plugins/sync.py
* change everything to ValidationError
* delegate destroy to super
* no request to repository url if can't install
* make the if cleaner
* add clarifying line
* add clarifying line
* fix url field type
* rename get_redis_instance to get_client, move to posthog.redis
* remove duplicate validation
* flip if around
* simplify api logic
* simplify plugin_config api, fix global_plugin error
* remove unnecessary field rename
* mypy
* Plugins UI (#2090)
* base UI
* more UI
* load plugin image if available
* toggle enabled plugin
* plugin cards for available
* custom plugins
* change plugin configuration to drawer
* asks for confirmation when enabling or disabling a plugin
* loading state
* separation of concerns, leave new styles for separate PR
* general improvements
* remove button when installation is not available
* preemptively avoid merge conflict with #2114
* move papercups widget & hide bottom bar when drawer is open
* allow clicking the entire plugin card
* address all feedback
* move plugins under "project" menu
* Hide "configure" from globally enabled plugins
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* add plugins opt in toggle to project/plugins
* choose pipeline based on team setting
* add "beta"
* plugin opt-in opt-out pages
* adjust install button
* remove tasks that are never called, remove PLUGINS_ENABLED global key
* fix responsive card display
* fix typo and drawer width
* skeleton fixes
* typo
* use "posthog-plugin-server" npm package
* "posthog-plugin-server" doc
* require the plugin server to be online before enabling plugins
* remove a few needeless "?." cases
* add hint for config_schema
* add hint for errors
* show plugin errors
* stop clicks if clicking on error
* show plugin errors
* loading indicators
* reload plugins when opting in/out
* nicer beta tag
* add frontend type
* fix mypy error
* fix test
* disable plugins if MULTI_TENANCY
* upgrade plugin-server version
* save event with plugin error
* upgrade plugin server
* squashed & optimized migrations
* remove unused import
* updates opt-in copy & hides tech details for cloud version
* fix cypress tests
* compare with None
* change plugins url and add redirect
* remove ellipsis
* use code snippet in plugin errors
* change github regex
* fix loading flickering on installing plugins
* add comment to plugin archive
* fix python style
* remove pip-tools (relic from the python plugin era)
* hard pin plugin server version
* remove copying of posthog.json from dev dockerfile (breaks if file doesn't exist, copied later anyway)
* update lockfile
Co-authored-by: James Greenhill <fuziontech@gmail.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-11-02 15:08:30 +01:00
|
|
|
#!/bin/bash
|
2020-12-14 16:05:18 +01:00
|
|
|
|
2020-12-10 07:29:06 +01:00
|
|
|
# Keys to export from django settings.py
|
2020-12-21 21:57:29 +01:00
|
|
|
KEYS="DATABASE_URL REDIS_URL PLUGINS_CELERY_QUEUE CELERY_DEFAULT_QUEUE KAFKA_HOSTS KAFKA_ENABLED BASE_DIR PLUGINS_RELOAD_PUBSUB_CHANNEL"
|
2020-12-09 18:08:25 +01:00
|
|
|
|
2020-12-10 07:29:06 +01:00
|
|
|
# Export $CONFIG to posthog-plugin-server
|
|
|
|
export CONFIG=`python manage.py print_settings --format json --indent 0 $KEYS 2> /dev/null | tr -d '\n' | sed 's/\n$//'`
|
2020-12-09 18:08:25 +01:00
|
|
|
|
|
|
|
# On _Heroku_, the $WEB_CONCURRENCY env contains suggested number of workers per dyno.
|
|
|
|
# Unfortunately we are running a NodeJS app, yet get the value for the "python" buildpack.
|
|
|
|
# Thus instead of using this env directly, calculate the real concurrency from $DYNO_RAM.
|
|
|
|
# Python: https://github.com/heroku/heroku-buildpack-python/blob/main/vendor/WEB_CONCURRENCY.sh
|
|
|
|
# NodeJS: https://devcenter.heroku.com/articles/node-concurrency#common-runtime
|
|
|
|
if [[ -n "${DYNO_RAM}" ]]; then
|
|
|
|
# One worker for 512MB of RAM, rounding up
|
|
|
|
export WORKER_CONCURRENCY=$(( ($DYNO_RAM - 1) / 512 + 1 ))
|
|
|
|
fi
|
|
|
|
|
2020-12-10 07:29:06 +01:00
|
|
|
cd plugins
|
|
|
|
|
2020-12-17 03:06:56 +01:00
|
|
|
yarn --frozen-lockfile --verbose
|
2020-12-17 01:35:19 +01:00
|
|
|
|
2020-12-10 07:29:06 +01:00
|
|
|
if [ $? -ne 0 ]; then
|
|
|
|
echo "Yarn failed!"
|
|
|
|
exit 1
|
|
|
|
fi
|
|
|
|
|
|
|
|
while true; do
|
|
|
|
# Run: posthog-plugin-server
|
2020-12-17 02:14:57 +01:00
|
|
|
yarn start
|
2020-12-10 07:29:06 +01:00
|
|
|
echo ""
|
|
|
|
echo "🔴 Plugin server crashed!"
|
|
|
|
echo "⌛️ Waiting 2 seconds before restarting!"
|
|
|
|
echo ""
|
|
|
|
sleep 2
|
|
|
|
done
|