Plugin v8 (#1946)
* plugin progress
* blah
* add posthog config for plugins
* test gitignore
* new functionality for plugins
* support local plugin paths
* also ignore symlinks
* add positional argument
* fixes
* small fixes
* config polish
* config passed to posthog plugin
* ooooooops
* symlink fix
* cleanse dir before loading
* add cache to plugins
* pickle the goods
* unlink symlink
* pass full config
* unlink even if link points to nothing
* fix fix
* return none if value is empty
* plugin model
* plugins scene
* add config schema to plugins
* install plugins
* save descriptions
* show descriptions
* edit plugin
* save plugin config
* plugin modal
* uninstall plugins
* UX cleanup
* add "required" to plugin config
* open plugin modal after install
* split to subcomponents
* install custom plugins
* rework backend for model plugins
* Plugins on models
* simple reload pubsub
* fix apps not installed
* fix master/main issue
* fix reload command
* use the github api to get the default branch
* init plugins only if not running migrate/makemigrations
* store plugins zip archives in postgres
* tag plugins to specific versions
* save plugins in pluginConfig
* update pluginConfigs instead of adding new rows, remove from redux on uninstall
* remove debug
* run plugins from db by team
* reload when deleting
* remove debug
* smarter handling of dynamic plugins, support local plugins again
* improve typings, add some nicer warnings
* yarn lock file after merge
* squash migrations and add "locked" field to plugins
* error if folder not found in zip
* unregister plugins
* skip plugin init in test mode
* basic plugin test
* avoid mutating the same prop hash
* add pip tools to requirements.txt
* fix mypy, fix manage.py script error
* avoid plugins with mypy
* mypy fix
* abstract redis into plugin and add team_id to reload
* refactor and start work on syncing with posthog.json
* start testing plugin loading from json
* test plugin deletion
* test for syncing plugins from config
* complete and then test local json plugin sync
* test converting back and forward between an local and http path
* remove global plugin config from plugins array in posthog.json
* rename configSchema --> config_schema
* fix migration after merge
* rename from_cli to from_json
* mypy
* import pip after plugin loaded
* show error details
* raise exceptions visible to the frontend
* sync plugins on load
* access control to updating plugins from the web
* access control
* remove posthog.json from git
* test config schema from json
* if you can install via the web, you can also configure
* remove separate view access
* title as "Plugins" instead of "Installed Plugins" if we can't install ourselves and don't see the repository
* add self.team to plugin base class
* add instance_init method
* refactor into files
* sync global plugin config from json
* make global plugins work, add test
* global plugins in interface, make them take precedence over local plugins
* add comments to plugin base class
* reload/reset plugins before each test
* add error field to plugins
* add many plugin zips
* add many plugin zips, fix imports
* store errors on plugin object and test them
* fix types
* add null to error
* can be with any team ID in the test
* save problems running plugins in the plugin_config model
* try to create redis connection pool only once
* throw if no redis
* mypy
* get instance inside heartbeat and not top level
* try caching pubsub
* try pip install with -q
* install pip externally
* remove uuid and typing, now in stdlib
* more verbosity
* add pip back
* catch exceptions
* quiet and no input for pip
* check plugin reload every 10sec on new task
* fix type errors
* fix requirements error message
* use repository.json
* only load and reload plugins on workers
* rename task
* support local js plugins via py-mini-racer
* load js plugins from zip files
* extract jsplugin class and convert to syntax that uses global functions instead of initializing a class
* process events via grpc
* process events with the "posthog-plugins" queue to enable plugins
* remove old native python & mini racer plugin code
* default to false
* change env vars
* fix test
* remove grpc tools
* skip plugins in migrate.py scripts
* fix migration
* change output of settings debug banner to STDERR
* start posthog plugin server with worker
* try to fix python 3.7 test
* add fallback for the optional argument
* annoying CI test debug
* try to finally fix python 3.7 test
* here we go again
* move plugins under instance
* move plugins npm start into its own folder
* more console.log debugginf
* and again
* move plugins to separate script
* more prints
* fix test error
* docker config
* small fixes
* dckerfile fix
* reload plugins via pubsub, upgrade version
* plugins that support team setup code
* sync if made changes from config
* move plugins in menu
* require node 14 in heroku for better plugin support (namely ?. support)
* bump node version in dockerfiles
* update node versions for github actions
* update the concurrency for heroku workers
* update the concurrency for heroku workers (add link)
* Fix migrations after merge
* add ignore_result to process_event tasks
* fix: docker-preview run in parallel bug
* change order of commands
* remove separate plugins server conf script
* clarify intent
* revert castaway change
* add context to plugins/sync.py
* change everything to ValidationError
* delegate destroy to super
* no request to repository url if can't install
* make the if cleaner
* add clarifying line
* add clarifying line
* fix url field type
* rename get_redis_instance to get_client, move to posthog.redis
* remove duplicate validation
* flip if around
* simplify api logic
* simplify plugin_config api, fix global_plugin error
* remove unnecessary field rename
* mypy
* Plugins UI (#2090)
* base UI
* more UI
* load plugin image if available
* toggle enabled plugin
* plugin cards for available
* custom plugins
* change plugin configuration to drawer
* asks for confirmation when enabling or disabling a plugin
* loading state
* separation of concerns, leave new styles for separate PR
* general improvements
* remove button when installation is not available
* preemptively avoid merge conflict with #2114
* move papercups widget & hide bottom bar when drawer is open
* allow clicking the entire plugin card
* address all feedback
* move plugins under "project" menu
* Hide "configure" from globally enabled plugins
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* add plugins opt in toggle to project/plugins
* choose pipeline based on team setting
* add "beta"
* plugin opt-in opt-out pages
* adjust install button
* remove tasks that are never called, remove PLUGINS_ENABLED global key
* fix responsive card display
* fix typo and drawer width
* skeleton fixes
* typo
* use "posthog-plugin-server" npm package
* "posthog-plugin-server" doc
* require the plugin server to be online before enabling plugins
* remove a few needeless "?." cases
* add hint for config_schema
* add hint for errors
* show plugin errors
* stop clicks if clicking on error
* show plugin errors
* loading indicators
* reload plugins when opting in/out
* nicer beta tag
* add frontend type
* fix mypy error
* fix test
* disable plugins if MULTI_TENANCY
* upgrade plugin-server version
* save event with plugin error
* upgrade plugin server
* squashed & optimized migrations
* remove unused import
* updates opt-in copy & hides tech details for cloud version
* fix cypress tests
* compare with None
* change plugins url and add redirect
* remove ellipsis
* use code snippet in plugin errors
* change github regex
* fix loading flickering on installing plugins
* add comment to plugin archive
* fix python style
* remove pip-tools (relic from the python plugin era)
* hard pin plugin server version
* remove copying of posthog.json from dev dockerfile (breaks if file doesn't exist, copied later anyway)
* update lockfile
Co-authored-by: James Greenhill <fuziontech@gmail.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-11-02 15:08:30 +01:00
|
|
|
#!/bin/bash
|
2020-12-14 16:05:18 +01:00
|
|
|
|
2021-01-21 15:39:44 +01:00
|
|
|
while test $# -gt 0; do
|
|
|
|
case "$1" in
|
2023-06-19 13:15:17 +02:00
|
|
|
-h | --help)
|
|
|
|
echo "USAGE:"
|
|
|
|
echo " bin/plugin-server [FLAGS]"
|
|
|
|
echo " "
|
|
|
|
echo "FLAGS:"
|
|
|
|
echo " -h, --help Print this help information."
|
|
|
|
echo " --no-restart-loop Run without restart loop. Recommended when deferring resiliency to e.g. docker-compose."
|
|
|
|
exit 0
|
|
|
|
;;
|
|
|
|
--no-restart-loop)
|
|
|
|
NO_RESTART_LOOP='true'
|
|
|
|
shift
|
|
|
|
;;
|
|
|
|
*)
|
|
|
|
break
|
|
|
|
;;
|
2021-01-21 15:39:44 +01:00
|
|
|
esac
|
|
|
|
done
|
|
|
|
|
|
|
|
export BASE_DIR=$(dirname $(dirname "$PWD/${0#./}"))
|
2022-05-26 11:01:55 +02:00
|
|
|
|
2023-06-19 13:15:17 +02:00
|
|
|
export KAFKA_HOSTS=${KAFKA_HOSTS:-'kafka:9092'}
|
2020-12-09 18:08:25 +01:00
|
|
|
|
2023-06-16 11:17:32 +02:00
|
|
|
if [[ -n $INJECT_EC2_CLIENT_RACK ]]; then
|
|
|
|
# To avoid cross-AZ Kafka traffic, set KAFKA_CLIENT_RACK from the EC2 metadata endpoint.
|
|
|
|
# TODO: switch to the downwards API when https://github.com/kubernetes/kubernetes/issues/40610 is released
|
|
|
|
TOKEN=$(curl --max-time 0.1 -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")
|
|
|
|
export KAFKA_CLIENT_RACK=$(curl --max-time 0.1 -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/placement/availability-zone-id)
|
|
|
|
fi
|
|
|
|
|
2022-02-17 17:56:24 +01:00
|
|
|
./bin/migrate-check
|
2022-01-06 14:30:47 +01:00
|
|
|
|
2021-11-13 20:52:00 +01:00
|
|
|
cd plugin-server
|
|
|
|
|
|
|
|
if [[ -n $DEBUG ]]; then
|
|
|
|
echo "🧐 Verifying installed packages..."
|
2022-12-12 10:28:06 +01:00
|
|
|
pnpm i --frozen-lockfile
|
2021-11-13 20:52:00 +01:00
|
|
|
fi
|
2020-12-10 07:29:06 +01:00
|
|
|
|
|
|
|
if [ $? -ne 0 ]; then
|
2021-01-21 15:39:44 +01:00
|
|
|
echo "💥 Verification failed!"
|
|
|
|
exit 1
|
2020-12-10 07:29:06 +01:00
|
|
|
fi
|
|
|
|
|
2023-09-15 16:55:46 +02:00
|
|
|
if [[ -n $DEBUG ]]; then
|
|
|
|
if [[ -n $NO_WATCH ]]; then
|
|
|
|
cmd="pnpm start:devNoWatch"
|
|
|
|
else
|
|
|
|
cmd="pnpm start:dev"
|
|
|
|
fi
|
|
|
|
else
|
|
|
|
cmd="node dist/index.js"
|
|
|
|
fi
|
2022-05-05 15:53:14 +02:00
|
|
|
|
2021-01-21 15:39:44 +01:00
|
|
|
if [[ -n $NO_RESTART_LOOP ]]; then
|
|
|
|
echo "▶️ Starting plugin server..."
|
2022-07-21 15:32:38 +02:00
|
|
|
trap 'kill -TERM $child 2>/dev/null; while kill -0 $child 2>/dev/null; do sleep 1; done' EXIT
|
2022-08-23 16:03:13 +02:00
|
|
|
$cmd &
|
2022-05-05 15:53:14 +02:00
|
|
|
child=$!
|
|
|
|
wait $child
|
2021-01-21 15:39:44 +01:00
|
|
|
else
|
|
|
|
echo "🔁 Starting plugin server in a resiliency loop..."
|
|
|
|
while true; do
|
2022-08-23 16:03:13 +02:00
|
|
|
$cmd
|
2021-01-21 15:39:44 +01:00
|
|
|
echo "💥 Plugin server crashed!"
|
|
|
|
echo "⌛️ Waiting 2 seconds before restarting..."
|
|
|
|
sleep 2
|
|
|
|
done
|
|
|
|
fi
|