* add recordings to path query
* uncomment cache
* add clarifying comment
* works for start/end paths
* move to extra fields/properties
* add tests
* cleanup
* update ff name
* fix flaky test
* test and handle path_dropoff_key case
* Run queries against person_distinct_id2 when async migration is done
* Only write to clickhouse_person_unique_id topic if async migration is incomplete
* Update query snapshots
* Update plugin-server
* Adjust caching logic
* ci(e2e): manually split cypress tests to improve speed
This change simply lists the spec dir for e2e tests, chunks them into
5, and passes these specs to cypress explicitly.
I initially tried to use https://github.com/kamilkisiela/split-tests/blob/master/packages/cypress/README.md
but ran into
```
We invoked the function exported by `/home/runner/work/posthog/posthog/cypress/plugins/index.js`, but it threw an error.
TypeError: Cannot read property 'mochaFile' of null
at loadReports (/home/runner/work/posthog/posthog/node_modules/dist/cypress/src/index.js:14:44)
at module.exports (/home/runner/work/posthog/posthog/node_modules/dist/cypress/src/index.js:44:19)
at module.exports (/home/runner/work/posthog/posthog/cypress/plugins/index.js:27:14)
at /home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:96:12
at tryCatcher (/home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/util.js:16:23)
at Function.Promise.attempt.Promise.try (/home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/node_modules/bluebird/js/release/method.js:39:29)
at load (/home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:93:7)
at EventEmitter.<anonymous> (/home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/lib/plugins/child/run_plugins.js:204:5)
at EventEmitter.emit (events.js:315:20)
at process.<anonymous> (/home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/lib/plugins/util.js:19:22)
at process.emit (events.js:315:20)
at process.emit (/home/runner/.cache/Cypress/6.7.0/Cypress/resources/app/packages/server/node_modules/source-map-support/source-map-support.js:495:21)
at emit (internal/child_process.js:876:12)
at processTicksAndRejections (internal/process/task_queues.js:85:21)
```
And so decided to go for something "simpler"
See for details https://github.com/PostHog/posthog/runs/4443218344?check_suite_focus=true
* run on master, ensures node_modules cache in scope
* run deps install on master
* increase concurrency
* run yarn install in prep section
* move if to job
* still run yarn install
* refactor
* comments
* change name so we don't have to update requirements
* Update e2e.yml
Co-authored-by: Tim Glaser <tim@posthog.com>
* github action artifact cypress videos
* artifact logs
* Fix cypress video export
* Redirect all output
* wait for initial retention load
* specify retry in ci
* use bin/start instead of bin/docker-*
* capture django app output in gunicorn log
* use docker-worker and docker-server again
* Remove gunicorn capture output settings
* Create a new way to get distinct id queries thats gated by team_id
* Update most cases to use the new query
* Convert EVENT_JOIN_PERSON_SQL to new query
* Mostly convert GET_DISTINCT_IDS_BY_PERSON_ID_FILTER
* Mostly convert GET_DISTINCT_IDS_BY_PROPERTY_SQL
* Convert GET_PERSON_IDS_BY_FILTER
* Flag benchmarks
* Resolve circular imports
* Update a snapshot test
* Add a test for the new query logic
* dev(codespaces): add "lean in" docker configuration for codespaces
This changes creates a dedicated `docker-compose.yml` and `Dockerfile`
for codespaces and doesn't attempt to have compatability with the
existing docker config files. This is intentional and is designed to be
a close to what codespaces would consider the ideal, so as to understand
what great would look like.
At the moment it doesn't do, but could:
1. install python and node packages in the image
2. push this image up to dockerhub
3. specify this image to be used for caching. There is a "cacheFrom"
option in the [.devcontainer
reference](https://code.visualstudio.com/docs/remote/devcontainerjson-reference)
although I haven't checked how this works with using
`docker-compose.yml`. Perhaps we just need to add the `image:`
property to the `app` service :fingerscrossed:
4. specify default extensions and settings to be used.
* Use python 3.9, not the latest python 3 codespaces supports
* Add some basic python default settings and extensions
* add github codespaces image build workflow
* chore: update docker-compose cache from :cache tag
* chore: use docker cache-to inline
* add codespaces platform comment
* temp run on branch
* chore: fix flatforms format
* chore: update qemu comment
* chore: codespaces build use master image
* chore: target x86_64
* chore: just use image for codespaces, don't try to build
* add in python, node requirements
* install node
* install node
* uncomment node deps install
* use app network stack for all services
* move .env to .dev.env and us in docker-compose
* set WORKDIR to workspace
* pin apt packages
* pin versions, remove apt-key
* add yarn cache clean
* Fix hadolint warnings
* make codespace build on master, or a PR with codespaces-build label
* add missing &&
* dont bother with clickhouse install, its huge
* dont be cute with the networks, just expose explicitly
* fix network
* create container zsh
* correct .env path
* point app to the clickhouse container
* Also install saml deps
* ignore .env again
* libxml already installed
* correct conditional workflow comment
* use plugin-server in same repo
* remove plugins dir
* update docker files
* clean up docker files:
* visual separation
* run plugin server test stack
* build and push test image
* revert push image
* fix default yarn start
* another fix
* fix some more
* add plugin-server-prod script
* make executable
* update scripts:
* revert prod script
* fixes
* fix tests
* clean
* move build to docker
* build plugin-server in prod dockerfile
* use different entrypoint for e2e test
* fix bugs with ts-node-dev
* restore yarn.lock
* lost a few changes in the rebase
* fix dockerfile
Co-authored-by: James Greenhill <fuziontech@gmail.com>
* Unify SAML and non-saml tests workflows
* Include matrix name in cache key
* Prettify yaml
* Measure test duration time
* Install pytest-split
* Try out parallelism
* Update name
* Fixup yaml
* Upload artifacts
* Run FOSS tests separately
* Run mark.ee tests only once
* Set up needed files properly
* Run backend tests only on one python version
* Run only cloud tests for cloud repo
* Remove test_durations file for posthog/
* Skip setting up ch on foss tests
* Fixup cloud tests
* Cache .test_durations file
* Fix ternary expression
* Only upload artefacts if needed
* .test_durations change
* Default to 3.8.5 in tests
* Add a note
* Only run `test_migrations_are_null` on new migrations
* Rename `DashboardItem` to `SavedInsight`
* Add explanation to `test_migrations_are_null` script
* Rename `test_dashboard` to `test_saved_insight_model`
* Ditch `DashboardItem{ViewSet,Serializer}` altogether
* Fix refresh via `InsightSerializer`
* "SavedInsight" to just "Insight"
* Update file names
* Update insight.py
* Update dashboardLogic.test.ts
* Update migration
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
At the moment we're just running tests on pull requests. The immediate
reason I'm making this change is to ensure we are producing a code
coverage report for master, which allows us to produce a patch
difference of coverage.
Sidenote: I'm also hoping will also trigger annotations to start being
added to PRs highlighting untested code, which I thought should have
started with https://github.com/PostHog/posthog/pull/6082
