0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-12-01 12:21:02 +01:00
Commit Graph

84 Commits

Author SHA1 Message Date
Marius Andra
67453e5a88
feat(data-exploration): data exploration backend API with types (#13933)
* feat(data-exploration): backend json schema validation

* Update snapshots

* Update snapshots

* Update snapshots

* Update snapshots

* Update snapshots

* Update snapshots

* Update snapshots

* Update snapshots

* refactor

* switch to pydantic

* split into v2 events list query

* add backend query endpoint

* Update snapshots

* make events query work

* bring back HogQLExpression, inline all __root__ only classes in python, remove timestamp in old events query

* Update snapshots

* Update snapshots

* schema and mypy

* Update snapshots

* Update snapshots

* restore /fake/ orderBy (only "timestamp" and "-timestamp" are supported)

* Update snapshots

* Update snapshots

* Update snapshots

* raise if invalid json

* Update snapshots

* remove __root__ hack

* improve comments, in and out of quotes

* shrug

* future proofing

* comment

* rename to "run_events_query"

* simplify

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-01-27 15:30:28 +01:00
Karl-Aksel Puulmann
b33347205c
fix: Revert "chore(tests): add pytest-randomly plugin to randomize test run order" (#12335)
* Revert "chore(tests): add pytest-randomly plugin to randomize test run order (#12307)"

This reverts commit 313de83cb8.

* Update snapshots

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-19 13:30:16 +03:00
zackelan
313de83cb8
chore(tests): add pytest-randomly plugin to randomize test run order (#12307)
* chore(tests): add pytest-randomly plugin to randomize test run order

* Update snapshots

* Update snapshots

* Update snapshots

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-10-18 09:34:34 -07:00
Guido Iaquinti
8145b682bf
chore(numpy): upgrade from 1.21.4 to 1.23.3 (#12142) 2022-10-07 14:14:39 +02:00
Paul D'Ambra
046184d4f3
chore: nice diff for pytest (#11963) 2022-09-26 14:12:45 +01:00
Paul D'Ambra
802a3e183c
chore: update python redis (#11741)
* largest change

* fewer dependency changes

* update to use not deprecated zrange

* obey mypy
2022-09-08 18:31:06 +01:00
Guido Iaquinti
49f330ee67
chore(deps): update flake8 dependencies (#11711) 2022-09-07 17:24:00 +02:00
Michael Matloka
5d2ad6c7bb
chore(deps): Update black to 22.8.0 (#11596)
* chore(deps): Update `black` to `22.8.0`

* Format
2022-09-05 14:38:54 +02:00
Tiina Turban
49f3311a65
chore: nuke protobuf fully (#10932) 2022-08-03 15:41:44 +02:00
Guido Iaquinti
35729df018
fix(pip-tools): upgrade from 6.6.1 to 6.6.2 (#10174) 2022-06-07 15:12:01 +02:00
Ben White
57874f9db2
feat(exports): Dashboard / Insight exporting (#9830)
* Adds chromium / selenium for image exporting
* Added uploading of downloads folder to artefacts
* Adds ExportButton to generate desired asset
2022-05-27 14:31:17 +02:00
Paul D'Ambra
55f23ee80d
chore: upgrade pip tools (#9859)
* update pip tools

* no need to do two step pip install
2022-05-19 15:01:32 +01:00
Neil Kakkar
5d94e93784
test: retry flakey experiment tests on failure (#9015)
* test: retry flakey experiment tests on failure

* bump max runs
2022-03-15 11:08:35 +00:00
Karl-Aksel Puulmann
9769ef0b5d
Improve backend CI (update .test_timings, run against different CH version) (#8708)
* Make clickhouse server image version configurable

* Attempt to use a reusable workflow for backend tests

* Try a composite action

* Add missing shell: bash

* Temporarily remove a step

* Cache id parameter

* Include the action when needing to rerun

* Move checking migrations to separate file

This didn't work in composite actions tests and slows down every
parallelized test execution for no reason. This should have been
separate in the first place.

* Rename job

* Improve check-migrations

* Start stack in the new job

* Remove shell

* Solve flaky test

* Remove unused file

* Create a test timings job

* Use checkout@v2 - https://github.com/cds-snc/github-actions/issues/23

* Ignore coverage files

* Save backend test durations

* Install pytest-split

* Clean up ci-backend

* Move actions code around

* wording improvement

* Save backend test durations

* Add a comment

* Make it possible to run tests with a different clickhouse version

* Remove debugging code

* Fix typo

* Boolean inputs please work

Fixes overeager test running as per https://github.com/actions/runner/issues/1483

Co-authored-by: PostHog Bot <hey@posthog.com>
2022-02-22 17:06:04 +02:00
Guido Iaquinti
ddc260e1f1
Remove flake8-commas (#8695) 2022-02-18 17:20:55 +01:00
Guido Iaquinti
7d22eebf03
Pin versions in requirements-dev.in (#8665) 2022-02-17 10:19:55 +00:00
Guido Iaquinti
40fe3a6f54
Rebuild requirements files (#8648) 2022-02-17 10:54:48 +01:00
Tim Glaser
041402e394
Update mypy and typed_ast (#8160)
* Update black to 21.12b0

* Don't update black, update mypy instead

* fix mypy issue
2022-01-20 14:19:43 +00:00
Neil Kakkar
b805a7e67f
Funnel multivariate experiment results API (#7922)
* funnel multivariate experiments

* cleanup

* remove numpy from requirements
2022-01-06 15:46:11 +00:00
Marius Andra
1696ed5950
Update mypy and typed-ast, fail early if db version not in range (#7599)
* error if unsupported db version

* upgrade mypy

* fix various types for mypy

* you can have it in any color you want, as long as it's black

* fix mypy

* Update `kafka-python`

* Format with Black

* Fix mypy after merge

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-12-10 09:29:04 +01:00
Paul D'Ambra
50b12d9bb7
Set debug, test, and primary_db env variables based on context (#7250)
* set debug, test, and primary_db env variables based on context

* remove unnecessary print

* also use pytest.ini for some tests run straight from pytest at the CLI
2021-11-20 17:18:39 +00:00
Karl-Aksel Puulmann
e66fbf0278
CI: Run EE tests in 5 minutes (#6759)
* Unify SAML and non-saml tests workflows

* Include matrix name in cache key

* Prettify yaml

* Measure test duration time

* Install pytest-split

* Try out parallelism

* Update name

* Fixup yaml

* Upload artifacts

* Run FOSS tests separately

* Run mark.ee tests only once

* Set up needed files properly

* Run backend tests only on one python version

* Run only cloud tests for cloud repo

* Remove test_durations file for posthog/

* Skip setting up ch on foss tests

* Fixup cloud tests

* Cache .test_durations file

* Fix ternary expression

* Only upload artefacts if needed

* .test_durations change

* Default to 3.8.5 in tests

* Add a note
2021-11-01 17:23:23 +02:00
Karl-Aksel Puulmann
457e151f58
Push person predicates down (#6346)
* Refactor column_optimizer to work differently

* WIP: Use counter over set

* Handle person filters in person query

* Remove a dead argument

* Use enum over parameter for determining behavior

* Allow excluding person properties mode when handled in person query

* Fix _get_person_query type

* Use correct table for funnel_event_query

* Remove unneeded override

* Add extra typing

* Filter by entity.properties in person query for trends

* Handle error 184 due to naming clash

* Better default for prop_filter_json_extract

* Update column_optimizer tests for Counter

* Handle person_props as extra_fields

* Handle breakdowns and person property filter pushdown

* Transform values correctly

* Simplify get_entity_filtering_params

* Fix funnel correlations

* Solve caching issues in trend people queries

* Remove @skip test

* Add syrupy tests for parse_prop_clauses

Can update these via --snapshot-update

* Add snapshot tests for person queries

* Add a few notes

* Update test to avoid collision

* Kill dead code

* Handle PR comments

* Update ee/clickhouse/queries/person_query.py

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
2021-10-13 14:00:47 +00:00
Harry Waye
c450be93f5
ci(tests): show untested code in PR with codecov (#6082) 2021-10-04 10:37:15 +01:00
Harry Waye
c07be4f0f6
chore(dev): make sure dev deps don't change production dep versions (#5945)
Previously it's possible that installing dev dependencies would end up
installing different versions to those used in production. Using `-c
requirements` we make sure that we constrain the dev deps by the
production pinned versions.

See
https://github.com/jazzband/pip-tools#workflow-for-layered-requirements
for details
2021-09-16 09:27:11 +01:00
Tim Glaser
92bd10ffa5
Speed up EE tests by not destroying tables (#5715)
* Speed up EE tests by not destroying tables

* Fix broken tests

* Speed up tests by not running migrations

* Fix last test

* req file?

* caching

* Parallel

* revert exp

* debug without parallel

* Revert "debug without parallel"

This reverts commit 83b3ad633f.

* Undo parallel tests

* Speed up more tests

* use final

* correct final

* print exception

* more prints

* move test

* add filter

* remove duplicate test

* prints here

* test wait

* set variable in query

* remove mutations_sync

* cleanup

Co-authored-by: eric <eeoneric@gmail.com>
2021-08-25 10:58:09 +02:00
Michael Matloka
420d8000c1
Support href/text element filters (#5654)
* Fix `icontains` and support `href/`text` element filters

* Test `is_set`/`is_not_set` too

* Require newer `flake8` to support Py 3.8 walrus operator

* Better handle empty mutlivalue operators and support doublequotes

* Polish all element filtering cases

* Roll back general `icontains` fix

* Make minor improvements
2021-08-20 13:53:15 +02:00
Paolo D'Amico
d0c93fa589
Upgrade to Django 3.1.12, urllib & requests (#4639) 2021-06-08 14:19:32 -07:00
Paolo D'Amico
14446c9ca7
Remove Django debug toolbar (#4029) 2021-04-26 10:02:45 -07:00
Paolo D'Amico
7ea9eb692c
Update Django to 3.1.8 (and related) (#4007) 2021-04-19 12:59:10 -07:00
Karl-Aksel Puulmann
1adaa8abb7
Chunk session recording events (#3705)
* Chunk session recording events

Closes https://github.com/PostHog/posthog/issues/3632 and replaces https://github.com/PostHog/posthog/pull/3566/files

This should make it possible to ingest large full snapshot events

Base64 is used to compress the data for serialization purposes.

pytest.mock is used for clean patching methods

* Mock time.time for py3.7 compatibility

* Group captured $snapshot events by $session_id

* Don't chunk already chunked payloads
2021-03-23 11:39:13 +01:00
Karl-Aksel Puulmann
24de8b55e4
Run tests via pytest (#3417)
* Compile requirements-dev.txt with latest pip-tools

* Install pytest

* Avoid picking up factories as tests

* New runner

* Always set TEST env variable running tests

Some of our tests rely on it.

* Remove repetition

* Fix a broken test

* Cut down noise from bin/tests

* Rename test factory

* Fix stickiness filter

* Skip a broken test

This has been broken since numpy removal PR. Sadly tests were not
running for this submodule

* Fix import on ee

* Run ee tests properly

The django_db_setup fixture will be automatically run when running ee/
module tests.

* Make tests run on CI

* Include REDIS_URL, fix cloud

* Set TEST env variable

* Hack cloud tests to work

* Attempt at workflow fix

* Import Person model when running ee tests

This module implicitly adds hooks, so this is needed when running tests

* Respect reuse-db for clickhouse

* Add custom markers to avoid warnings

* pytest: use ch test database always

Accidentally wiped by ch setup a few times without this. Oops

* Remove repetition in tests

* Pytest: Always run migrations

Testing a state cleanup fix

* Use same DB in conftest and main code

* Pytest: autoset TEST setting without env variable

* fix broken test

Co-authored-by: eric <eeoneric@gmail.com>
2021-02-24 09:32:44 +02:00
James Greenhill
7ac64c84f1
Basic caching for Clickhouse to redis (#2300)
* Basic caching for Clickhouse to redis

* Use redis for caching results

* add tests and fix bugs

* fix mypy

* add fakeredis as req

* add fakeredis to github action for testing

* add fakeredis to cloud tests too

* pickle -> json

* bytes

* json in tests

* tuplefy
2020-11-13 13:20:00 +01:00
Michael Matloka
b7b9e8caf3
Remove some seemingly unused requirements and use pip-compile (#2192)
* Remove some seemingly unused requirements

* Remove a few more

* Bring back dependencies of other requirements

* pip-compile prod requirements

* Sync with master updates

* Fix grep

* Revert "Fix grep"

This reverts commit 2e6538cad4.

* Fix grep but using cut

* Clean up and regenerate requirements

* Simplify requirements-dev

* Fix fakeredis import in production

* Black
2020-11-10 10:17:48 +01:00