0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-12-01 12:21:02 +01:00
posthog/bin/plugin-server

71 lines
2.4 KiB
Plaintext
Raw Normal View History

Plugin v8 (#1946) * plugin progress * blah * add posthog config for plugins * test gitignore * new functionality for plugins * support local plugin paths * also ignore symlinks * add positional argument * fixes * small fixes * config polish * config passed to posthog plugin * ooooooops * symlink fix * cleanse dir before loading * add cache to plugins * pickle the goods * unlink symlink * pass full config * unlink even if link points to nothing * fix fix * return none if value is empty * plugin model * plugins scene * add config schema to plugins * install plugins * save descriptions * show descriptions * edit plugin * save plugin config * plugin modal * uninstall plugins * UX cleanup * add "required" to plugin config * open plugin modal after install * split to subcomponents * install custom plugins * rework backend for model plugins * Plugins on models * simple reload pubsub * fix apps not installed * fix master/main issue * fix reload command * use the github api to get the default branch * init plugins only if not running migrate/makemigrations * store plugins zip archives in postgres * tag plugins to specific versions * save plugins in pluginConfig * update pluginConfigs instead of adding new rows, remove from redux on uninstall * remove debug * run plugins from db by team * reload when deleting * remove debug * smarter handling of dynamic plugins, support local plugins again * improve typings, add some nicer warnings * yarn lock file after merge * squash migrations and add "locked" field to plugins * error if folder not found in zip * unregister plugins * skip plugin init in test mode * basic plugin test * avoid mutating the same prop hash * add pip tools to requirements.txt * fix mypy, fix manage.py script error * avoid plugins with mypy * mypy fix * abstract redis into plugin and add team_id to reload * refactor and start work on syncing with posthog.json * start testing plugin loading from json * test plugin deletion * test for syncing plugins from config * complete and then test local json plugin sync * test converting back and forward between an local and http path * remove global plugin config from plugins array in posthog.json * rename configSchema --> config_schema * fix migration after merge * rename from_cli to from_json * mypy * import pip after plugin loaded * show error details * raise exceptions visible to the frontend * sync plugins on load * access control to updating plugins from the web * access control * remove posthog.json from git * test config schema from json * if you can install via the web, you can also configure * remove separate view access * title as "Plugins" instead of "Installed Plugins" if we can't install ourselves and don't see the repository * add self.team to plugin base class * add instance_init method * refactor into files * sync global plugin config from json * make global plugins work, add test * global plugins in interface, make them take precedence over local plugins * add comments to plugin base class * reload/reset plugins before each test * add error field to plugins * add many plugin zips * add many plugin zips, fix imports * store errors on plugin object and test them * fix types * add null to error * can be with any team ID in the test * save problems running plugins in the plugin_config model * try to create redis connection pool only once * throw if no redis * mypy * get instance inside heartbeat and not top level * try caching pubsub * try pip install with -q * install pip externally * remove uuid and typing, now in stdlib * more verbosity * add pip back * catch exceptions * quiet and no input for pip * check plugin reload every 10sec on new task * fix type errors * fix requirements error message * use repository.json * only load and reload plugins on workers * rename task * support local js plugins via py-mini-racer * load js plugins from zip files * extract jsplugin class and convert to syntax that uses global functions instead of initializing a class * process events via grpc * process events with the "posthog-plugins" queue to enable plugins * remove old native python & mini racer plugin code * default to false * change env vars * fix test * remove grpc tools * skip plugins in migrate.py scripts * fix migration * change output of settings debug banner to STDERR * start posthog plugin server with worker * try to fix python 3.7 test * add fallback for the optional argument * annoying CI test debug * try to finally fix python 3.7 test * here we go again * move plugins under instance * move plugins npm start into its own folder * more console.log debugginf * and again * move plugins to separate script * more prints * fix test error * docker config * small fixes * dckerfile fix * reload plugins via pubsub, upgrade version * plugins that support team setup code * sync if made changes from config * move plugins in menu * require node 14 in heroku for better plugin support (namely ?. support) * bump node version in dockerfiles * update node versions for github actions * update the concurrency for heroku workers * update the concurrency for heroku workers (add link) * Fix migrations after merge * add ignore_result to process_event tasks * fix: docker-preview run in parallel bug * change order of commands * remove separate plugins server conf script * clarify intent * revert castaway change * add context to plugins/sync.py * change everything to ValidationError * delegate destroy to super * no request to repository url if can't install * make the if cleaner * add clarifying line * add clarifying line * fix url field type * rename get_redis_instance to get_client, move to posthog.redis * remove duplicate validation * flip if around * simplify api logic * simplify plugin_config api, fix global_plugin error * remove unnecessary field rename * mypy * Plugins UI (#2090) * base UI * more UI * load plugin image if available * toggle enabled plugin * plugin cards for available * custom plugins * change plugin configuration to drawer * asks for confirmation when enabling or disabling a plugin * loading state * separation of concerns, leave new styles for separate PR * general improvements * remove button when installation is not available * preemptively avoid merge conflict with #2114 * move papercups widget & hide bottom bar when drawer is open * allow clicking the entire plugin card * address all feedback * move plugins under "project" menu * Hide "configure" from globally enabled plugins Co-authored-by: Marius Andra <marius.andra@gmail.com> * add plugins opt in toggle to project/plugins * choose pipeline based on team setting * add "beta" * plugin opt-in opt-out pages * adjust install button * remove tasks that are never called, remove PLUGINS_ENABLED global key * fix responsive card display * fix typo and drawer width * skeleton fixes * typo * use "posthog-plugin-server" npm package * "posthog-plugin-server" doc * require the plugin server to be online before enabling plugins * remove a few needeless "?." cases * add hint for config_schema * add hint for errors * show plugin errors * stop clicks if clicking on error * show plugin errors * loading indicators * reload plugins when opting in/out * nicer beta tag * add frontend type * fix mypy error * fix test * disable plugins if MULTI_TENANCY * upgrade plugin-server version * save event with plugin error * upgrade plugin server * squashed & optimized migrations * remove unused import * updates opt-in copy & hides tech details for cloud version * fix cypress tests * compare with None * change plugins url and add redirect * remove ellipsis * use code snippet in plugin errors * change github regex * fix loading flickering on installing plugins * add comment to plugin archive * fix python style * remove pip-tools (relic from the python plugin era) * hard pin plugin server version * remove copying of posthog.json from dev dockerfile (breaks if file doesn't exist, copied later anyway) * update lockfile Co-authored-by: James Greenhill <fuziontech@gmail.com> Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-11-02 15:08:30 +01:00
#!/bin/bash
while test $# -gt 0; do
case "$1" in
-h|--help)
echo "USAGE:"
echo " bin/plugin-server [FLAGS]"
echo " "
echo "FLAGS:"
echo " -h, --help Print this help information."
echo " --no-restart-loop Run without restart loop. Recommended when deferring resiliency to e.g. docker-compose."
exit 0
;;
--no-restart-loop)
NO_RESTART_LOOP='true'
shift
;;
*)
break
;;
esac
done
if [[ -n $POSTHOG_DB_NAME ]]; then
[[ -n $POSTHOG_DB_PASSWORD ]] && DB_PASSWORD_COMPONENT=":$POSTHOG_DB_PASSWORD"
export DATABASE_URL="postgres://${POSTHOG_DB_USER:-"postgres"}${DB_PASSWORD_COMPONENT}@${POSTHOG_POSTGRES_HOST:-"localhost"}:${POSTHOG_POSTGRES_PORT:-"5432"}/${POSTHOG_DB_NAME}"
fi
if [[ -z $REDIS_URL ]]; then
[[ -n $POSTHOG_REDIS_PASSWORD ]] && REDIS_PASSWORD_COMPONENT=":$POSTHOG_REDIS_PASSWORD@"
export REDIS_URL="redis://${REDIS_PASSWORD_COMPONENT}${POSTHOG_REDIS_HOST:-"localhost"}:${POSTHOG_REDIS_PORT:-"6379"}/"
fi
export BASE_DIR=$(dirname $(dirname "$PWD/${0#./}"))
# Crudely extract netlocs by removing "kafka://" in a compatibility approach
export KAFKA_HOSTS=${KAFKA_HOSTS:-$(echo $KAFKA_URL | sed -e "s/kafka\(\+ssl\)\{0,1\}:\/\///g")}
# Check PRIMARY_DB in a compatibility approach
export KAFKA_ENABLED=${KAFKA_ENABLED:-$([[ $PRIMARY_DB = "clickhouse" ]] && echo "true" || echo "false")}
# On _Heroku_, the $WEB_CONCURRENCY env contains suggested number of workers per dyno.
# Unfortunately we are running a NodeJS app, yet get the value for the "python" buildpack.
# Thus instead of using this env directly, calculate the real concurrency from $DYNO_RAM.
# Python: https://github.com/heroku/heroku-buildpack-python/blob/main/vendor/WEB_CONCURRENCY.sh
# NodeJS: https://devcenter.heroku.com/articles/node-concurrency#common-runtime
if [[ -n $DYNO_RAM ]]; then
# One worker for 512MB of RAM, rounding up
export WORKER_CONCURRENCY=$(( ($DYNO_RAM - 1) / 512 + 1 ))
fi
cd plugins
echo "🧐 Verifying installed packages..."
yarn --frozen-lockfile
if [ $? -ne 0 ]; then
echo "💥 Verification failed!"
exit 1
fi
if [[ -n $NO_RESTART_LOOP ]]; then
echo "▶️ Starting plugin server..."
yarn start
else
echo "🔁 Starting plugin server in a resiliency loop..."
while true; do
yarn start
echo "💥 Plugin server crashed!"
echo "⌛️ Waiting 2 seconds before restarting..."
sleep 2
done
fi