allow asgi/wsgi to be configurable by env var
This will let us roll out asgi separate across services as we had
issues with our recordings capture pods on asgi
* test: Regular checkup of visual regression tests
* Fix billing gauge animation
Animations done in JS can't be stopped automatically by the Storybook test runner. CSS animations can, easily, and they are anyway the cleaner and more performant way of achieving the same here.
* Rename misleading feature flag `recentInsights` to `relatedInsights`
* Mock homepage endpoints to avoid error toasts
* Wait for the recordings list in the notebook node story
* Fix `featureFlagLogic`
* Wait for `.NotebookNode__content`
* Try to optimize
* Screenshot failures and upload as artifacts
* Fix remaining failures
* Increase timeouts
* Fix rendering of Survey stories
* Remove `clang-format`
* Update UI snapshots for `chromium` (2)
* Update UI snapshots for `chromium` (2)
* Update UI snapshots for `chromium` (2)
* Fix alignment of series name in insights details
* Try to fix experiment story flakiness
* Include toasts in loaders
* Fix superfluous toast
* Fix un-awaited breakpoints
* Update UI snapshots for `chromium` (1)
* Update UI snapshots for `chromium` (1)
* Update UI snapshots for `chromium` (1)
* Make login snapshots slightly stabler
* Update UI snapshots for `chromium` (2)
* Update UI snapshots for `chromium` (2)
* Skip incorrect Surveys story
* Update UI snapshots for `chromium` (2)
* Revert msw upgrade
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Add partial C++ parser of HogQL
* Support all the rules and add proper error handling
* Use `AlignAfterOpenBracket: BlockIndent`
* Reuse all the parser tests for the C++ backend
* Fix relationship between headers and implementations
* Add more error handling and fix minor issues
* Build both Python and C++ parsers in the package.json script
* Update ARRAY JOIN error assertion
* Improve timeit
* Move the C extension to the top level
* Refactor `vector_to_list_string`
* Build the parser on Linux
* Build wheels for the parser
* Simplify Linux build and fix macOS
* Fix Homebrew paths on x86 and don't fail fast in CI
* Set MACOSX_DEPLOYMENT_TARGET for C++20
* Set up QEMU for Linux ARM builds
* Publish the wheels on PyPI
* Avoiding Linux ARM emulation in CI for now
* Build sdist too
* Revert Dockerfile changes
* Fix PyPI publish
* Add README and optimize sdist build
* Use setup.py directly instead of build
* Use PyPI hogql-parser instead of local
* Also revert production-unit.Dockerfile
* Fix sdist upload and add Linux ARM back
* No Linux ARM build in the end
* Fix artifact uploading
* Do try building Linux ARM
We need this for prod.
* Use `npm` in `grammar:build`
`pnpm` is not available in that job.
* Fix formatting of hogql_parser
* Build everything on macOS
* Revert "Build everything on macOS"
Not so fast actually.
* Use hogql-parser=0.1.1
* Fix dylib in macOS wheel
* Bump hogql-parser version
* Fix missing module error
* Delete timeit.py
* Make error handling robust
* Format the C++
* Use `hogql-parser==0.1.1`
* Fix reserved keyword error assertions
* Use HEAD hogql_paresr in CI
* Fix `apt` usage
* Add some sudo in CI
* Ensure package will be releasable before build
* Bump version to 0.1.3
* Cover C++ `unquote_string` with tests
* Use BuildJet ARM runners for ARM builds
* Add some instructions
* Add HogQL version check to backend CI
* Update requirements.txt
* Use `setuptools` instead of the deprecated `distutils`
* Fix working dir in backend CI
* Align ANTLR versions
* Add test for "mismatched input"
This is thrown differently than other HogQLSyntaxExceptions in C++, so might help reveal what's going on with tests failing only on Linux CI and not macOS dev
* Add types and bump version
* Comment instead of failing version check
* Automate hogql-release version bump
* Fix checkout token
* Don't build hogql-parser if there were no changes
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Improve documentation
* Use new hogql-parser version
* Fix error start and end initialization
* Note `antlr4-cpp-runtime`
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Also remove NUL chars in C++
* Check ANTLR4 runtime archive checksum for security
* Note more decrefs to add
* Add vector size checks
* Use new hogql-parser version
* Don't support the `start` arg in C++ `parse_expr`
* Use new hogql-parser version
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Add docker image that uses nginx unit instead of gunicorn
🦄🔫
* Add unit build to CI
* Fix duplicate id
* try 3.11
* Only build for amd64
need python3.11 for unit image on arm
* chore: use pnpm to manage dependencies
* Fix CI errors
* Don't report Docker image size for external PRs
* Fix pnpm-lock.yaml formatting
* Fix module versions
* Ignore pnpm-lock.yaml
* Upgrade Cypress action for pnpm support
* Set up node and pnpm before Cypress
* Fix typescript issues
* Include patches directory in Dockerfile
* Fix Jest tests in CI
* Update lockfile
* Update lockfile
* Clean up Dockerfile
* Update pnpm-lock.yaml to reflect current package.json files
* remove yarn-error.log from .gitignore
* formatting
* update data exploration readme
* type jest.config.ts
* fix @react-hook issues for jest
* fix react-syntax-highlighter issues for jest
* fix jest issues from query-selector-shadow-dom
* fix transform ignore patterns and undo previous fixes
* add missing storybook peer dependencies
* fix nullish coalescing operator for storybook
* reorder storybook plugins
* update editor-update-tsd warning to new npm script
* use legacy ssl for chromatic / node 18 compatibility
* use pnpm for visual regression testing workflow
* use node 16 for chromatic
* add @babel/plugin-proposal-nullish-coalescing-operator as direct dependency
* try fix for plugin-server
* cleanup
* fix comment and warning
* update more comments
* update playwright dockerfile
* update plugin source types
* conditional image size reporting
* revert react-native instructions
* less restrictive pnpm verions
* use ref component name in line with style guide
Co-authored-by: Jacob Gillespie <jacobwgillespie@gmail.com>
* feat: add libmaxminddb0 as dependency. C library will speed things up significantly
* pin libmaxminddb to 1.5 for whats available from APK
* get geolite2 db during build
* add settings for geoip2 django contrib library
* black formatting
* consistently use share director
* isort fixes
* remove GeoLite2-City.mmdb from git and add script to ./bin/start to download it if file does not exist
* remove GeoLite2-City.mmdb from git
* add doc for share directory expaining why it exists
* relative path for curl in build
* shared vs share consistency
* Update snapshots
* brotli decompress
* ..everywhere
Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
Co-authored-by: neilkakkar <neilkakkar@users.noreply.github.com>
* dev(codespaces): use Dockerfile not docker-compose
I wasn't so confident around the updating of the docker-compose file to
include minio, and the workflow around it. I'd rather just drop into
something with docker-in-docker and run docker-compose myself.
Also finally added some extensions I'm using so I don't need to enable
them all the time.
TODO: update docker-compose file to work with docker-in-docker. At the
moment I don't think the networking will be correct.
* Merge remote-tracking branch 'origin/master' into hackaton_session_rec
* add new line
* remove yarn install, it will be overwritten by the codespaces mount
* revert compose formatting
* format with prettier
* use plugin-server in same repo
* remove plugins dir
* update docker files
* clean up docker files:
* visual separation
* run plugin server test stack
* build and push test image
* revert push image
* fix default yarn start
* another fix
* fix some more
* add plugin-server-prod script
* make executable
* update scripts:
* revert prod script
* fixes
* fix tests
* clean
* move build to docker
* build plugin-server in prod dockerfile
* use different entrypoint for e2e test
* fix bugs with ts-node-dev
* restore yarn.lock
* lost a few changes in the rebase
* fix dockerfile
Co-authored-by: James Greenhill <fuziontech@gmail.com>
* esbuild package
* almost get esbuild working
* fix react-virtualized imports
* add splitting
* fix funny import reorder bug
* fix squeakAudio referring to itself
* write index.html file
* fix some bad imports
* update antd paths
* remove raw-loader usage, it didn't work anyway
* refactor and copy public
* build app and toolbar
* get toolbar working, but without styles
* make toolbar and its styles work
* shared dashboards
* clean frontend build before rebuilding
* add watch mode
* reorder tasks
* revert js url
* incremental builds of app with debounced chokidar watching
* common build/watch script
* improve logs
* watch during firrst build
* fix toolbar url
* fix wrongly exported scene
* create sceneProxyLogic to untangle sceneLogic from all bundles
* disconnect sceneLogic and refactor setPageTitle
* live reloading server
* rename utils file
* only wait for /static
* fix encoding
* simplify
* add missing dayjs plugins
* fix pathless logics
* simplify options
* add jsx for webapck
* slight delay to catch changes
* a type is a type
* fix build
* esbuild in start
* funnelLogic path
* include all files with a "." (so .mjs, etc) in /frontend/ to docker
* rename to "utils.mjs", make "build.mjs" executable
* improve erroring
* revert some needless changes
* more reverts
* change some scripts
* remove setuff
* clarify function
* make "--host 0.0.0.0" work
* fix import order issue in webpack
* remove webpack css inlining for toolbar to simplify config
* make toolbar with external styles work in storybook
* move live server injection into django
* fix undefined bug
* simplify setup to work with injection directly in http://localhost:8000 (no proxying needed on :8234)
* add comments
* Fix `fse` usage
I was getting this otherwise:
$ node frontend/build.mjs
file:///Users/twixes/Developer/posthog/frontend/utils.mjs:46
fse.copySync(srcDir, destDir, { overwrite: true }, function (err) {
^
TypeError: fse.copySync is not a function
at copyPublicFolder (file:///Users/twixes/Developer/posthog/frontend/utils.mjs:46:9)
at file:///Users/twixes/Developer/posthog/frontend/build.mjs:5:1
at ModuleJob.run (internal/modules/esm/module_job.js:146:23)
at async Loader.import (internal/modules/esm/loader.js:165:24)
at async Object.loadESM (internal/process/esm_loader.js:68:5)
* Mock `process` for VFile used by ReactMarkdown
I was getting this otherwise:
core.js:55 Uncaught ReferenceError: process is not defined
at new VFile (core.js:55)
at VFile (core.js:49)
at Function.parse (index.js:273)
at ReactMarkdown2 (react-markdown.js:42)
at renderWithHooks (react-dom.development.js:14803)
at mountIndeterminateComponent (react-dom.development.js:17482)
at beginWork (react-dom.development.js:18596)
at HTMLUnknownElement.callCallback2 (react-dom.development.js:188)
at Object.invokeGuardedCallbackDev (react-dom.development.js:237)
at invokeGuardedCallback (react-dom.development.js:292)
* Mock `process.env` for VFile used by ReactMarkdown
I was getting this otherwise:
platform.ts:73 Uncaught TypeError: Cannot read properties of undefined (reading 'ENABLE_VSCODE_BROWSER_CODE_LOADING')
at platform.ts:73
at platform.ts:79
at Function.r._invokeFactory (loader.js:1118)
at r.complete (loader.js:1128)
at r._onModuleComplete (loader.js:1754)
at r._resolve (loader.js:1714)
at r.defineModule (loader.js:1357)
at _ (loader.js:1804)
at numbers.ts:10
at fake:1
* pass the heavy appScenes to sceneLogic through props via App.tsx
* remove sceneProxyLogic
* remove exported variables
* fix sceneLogic test
Co-authored-by: Michael Matloka <dev@twixes.com>
* Remove some seemingly unused requirements
* Remove a few more
* Bring back dependencies of other requirements
* pip-compile prod requirements
* Sync with master updates
* Fix grep
* Revert "Fix grep"
This reverts commit 2e6538cad4.
* Fix grep but using cut
* Clean up and regenerate requirements
* Simplify requirements-dev
* Fix fakeredis import in production
* Black
* plugin progress
* blah
* add posthog config for plugins
* test gitignore
* new functionality for plugins
* support local plugin paths
* also ignore symlinks
* add positional argument
* fixes
* small fixes
* config polish
* config passed to posthog plugin
* ooooooops
* symlink fix
* cleanse dir before loading
* add cache to plugins
* pickle the goods
* unlink symlink
* pass full config
* unlink even if link points to nothing
* fix fix
* return none if value is empty
* plugin model
* plugins scene
* add config schema to plugins
* install plugins
* save descriptions
* show descriptions
* edit plugin
* save plugin config
* plugin modal
* uninstall plugins
* UX cleanup
* add "required" to plugin config
* open plugin modal after install
* split to subcomponents
* install custom plugins
* rework backend for model plugins
* Plugins on models
* simple reload pubsub
* fix apps not installed
* fix master/main issue
* fix reload command
* use the github api to get the default branch
* init plugins only if not running migrate/makemigrations
* store plugins zip archives in postgres
* tag plugins to specific versions
* save plugins in pluginConfig
* update pluginConfigs instead of adding new rows, remove from redux on uninstall
* remove debug
* run plugins from db by team
* reload when deleting
* remove debug
* smarter handling of dynamic plugins, support local plugins again
* improve typings, add some nicer warnings
* yarn lock file after merge
* squash migrations and add "locked" field to plugins
* error if folder not found in zip
* unregister plugins
* skip plugin init in test mode
* basic plugin test
* avoid mutating the same prop hash
* add pip tools to requirements.txt
* fix mypy, fix manage.py script error
* avoid plugins with mypy
* mypy fix
* abstract redis into plugin and add team_id to reload
* refactor and start work on syncing with posthog.json
* start testing plugin loading from json
* test plugin deletion
* test for syncing plugins from config
* complete and then test local json plugin sync
* test converting back and forward between an local and http path
* remove global plugin config from plugins array in posthog.json
* rename configSchema --> config_schema
* fix migration after merge
* rename from_cli to from_json
* mypy
* import pip after plugin loaded
* show error details
* raise exceptions visible to the frontend
* sync plugins on load
* access control to updating plugins from the web
* access control
* remove posthog.json from git
* test config schema from json
* if you can install via the web, you can also configure
* remove separate view access
* title as "Plugins" instead of "Installed Plugins" if we can't install ourselves and don't see the repository
* add self.team to plugin base class
* add instance_init method
* refactor into files
* sync global plugin config from json
* make global plugins work, add test
* global plugins in interface, make them take precedence over local plugins
* add comments to plugin base class
* reload/reset plugins before each test
* add error field to plugins
* add many plugin zips
* add many plugin zips, fix imports
* store errors on plugin object and test them
* fix types
* add null to error
* can be with any team ID in the test
* save problems running plugins in the plugin_config model
* try to create redis connection pool only once
* throw if no redis
* mypy
* get instance inside heartbeat and not top level
* try caching pubsub
* try pip install with -q
* install pip externally
* remove uuid and typing, now in stdlib
* more verbosity
* add pip back
* catch exceptions
* quiet and no input for pip
* check plugin reload every 10sec on new task
* fix type errors
* fix requirements error message
* use repository.json
* only load and reload plugins on workers
* rename task
* support local js plugins via py-mini-racer
* load js plugins from zip files
* extract jsplugin class and convert to syntax that uses global functions instead of initializing a class
* process events via grpc
* process events with the "posthog-plugins" queue to enable plugins
* remove old native python & mini racer plugin code
* default to false
* change env vars
* fix test
* remove grpc tools
* skip plugins in migrate.py scripts
* fix migration
* change output of settings debug banner to STDERR
* start posthog plugin server with worker
* try to fix python 3.7 test
* add fallback for the optional argument
* annoying CI test debug
* try to finally fix python 3.7 test
* here we go again
* move plugins under instance
* move plugins npm start into its own folder
* more console.log debugginf
* and again
* move plugins to separate script
* more prints
* fix test error
* docker config
* small fixes
* dckerfile fix
* reload plugins via pubsub, upgrade version
* plugins that support team setup code
* sync if made changes from config
* move plugins in menu
* require node 14 in heroku for better plugin support (namely ?. support)
* bump node version in dockerfiles
* update node versions for github actions
* update the concurrency for heroku workers
* update the concurrency for heroku workers (add link)
* Fix migrations after merge
* add ignore_result to process_event tasks
* fix: docker-preview run in parallel bug
* change order of commands
* remove separate plugins server conf script
* clarify intent
* revert castaway change
* add context to plugins/sync.py
* change everything to ValidationError
* delegate destroy to super
* no request to repository url if can't install
* make the if cleaner
* add clarifying line
* add clarifying line
* fix url field type
* rename get_redis_instance to get_client, move to posthog.redis
* remove duplicate validation
* flip if around
* simplify api logic
* simplify plugin_config api, fix global_plugin error
* remove unnecessary field rename
* mypy
* Plugins UI (#2090)
* base UI
* more UI
* load plugin image if available
* toggle enabled plugin
* plugin cards for available
* custom plugins
* change plugin configuration to drawer
* asks for confirmation when enabling or disabling a plugin
* loading state
* separation of concerns, leave new styles for separate PR
* general improvements
* remove button when installation is not available
* preemptively avoid merge conflict with #2114
* move papercups widget & hide bottom bar when drawer is open
* allow clicking the entire plugin card
* address all feedback
* move plugins under "project" menu
* Hide "configure" from globally enabled plugins
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* add plugins opt in toggle to project/plugins
* choose pipeline based on team setting
* add "beta"
* plugin opt-in opt-out pages
* adjust install button
* remove tasks that are never called, remove PLUGINS_ENABLED global key
* fix responsive card display
* fix typo and drawer width
* skeleton fixes
* typo
* use "posthog-plugin-server" npm package
* "posthog-plugin-server" doc
* require the plugin server to be online before enabling plugins
* remove a few needeless "?." cases
* add hint for config_schema
* add hint for errors
* show plugin errors
* stop clicks if clicking on error
* show plugin errors
* loading indicators
* reload plugins when opting in/out
* nicer beta tag
* add frontend type
* fix mypy error
* fix test
* disable plugins if MULTI_TENANCY
* upgrade plugin-server version
* save event with plugin error
* upgrade plugin server
* squashed & optimized migrations
* remove unused import
* updates opt-in copy & hides tech details for cloud version
* fix cypress tests
* compare with None
* change plugins url and add redirect
* remove ellipsis
* use code snippet in plugin errors
* change github regex
* fix loading flickering on installing plugins
* add comment to plugin archive
* fix python style
* remove pip-tools (relic from the python plugin era)
* hard pin plugin server version
* remove copying of posthog.json from dev dockerfile (breaks if file doesn't exist, copied later anyway)
* update lockfile
Co-authored-by: James Greenhill <fuziontech@gmail.com>
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
* Whitelist kea-typegen folder in .dockerignore
* Added debug to devdockerfile
* quote style fix
* add .kearc to dockerignore (needed to generate types in yarn build)
* Add tsconfig.json to dockerfile
* add tsconfig and kearc to dockerfiles
* updated docker files to suppress secret key error
* use fake secret key in collectstatic
Co-authored-by: Yakko Majuri <yakko.majuri@gmail.com>
* ignore copying frontend/dist folder - otherwise whatever you build in docker will get overridden by local build artifacts if any exist in frontend/dist
* support configuring redis with POSTHOG_REDIS_HOST and other vars in addition to REDIS_URL
* remove "the next version" in worker requirement modal
* split beat and celery scripts
* remove chart folder
* celery heartbeat every 10sec, reduce distributed beat lock hold time
* remove dockerfile local link
* add localhost redis url for tests
