0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-22 08:40:03 +01:00
Commit Graph

47 Commits

Author SHA1 Message Date
ted kaemming
0944412f48
feat: Support marking materialized columns as disabled and dropping them (#26068) 2024-11-13 10:57:27 -08:00
Ben White
cc9cb45d58
feat: Action steps refactor (attempt 2) (#22137) 2024-05-07 15:25:19 +02:00
Ben White
0fc2c9eccd
revert: "feat: Action steps refactor (part1) (#22091)" (#22136) 2024-05-07 12:32:41 +02:00
Ben White
776298d2a8
feat: Action steps refactor (part1) (#22091) 2024-05-07 12:08:13 +02:00
Julian Bez
9576fab1e4
chore: Add Pyupgrade rules (#21714)
* Add Pyupgrade rules
* Set correct Python version
2024-04-25 08:22:28 +01:00
Thomas Obermüller
2ade0951de
refactor(style): do not use star imports for locals (#20915) 2024-03-14 13:55:52 +01:00
Tom Owers
5513be7731
chore: upgraded Ruff linter (#18188)
* Upgraded Ruff linter

* Formatted whole codebase with new ruff rules

* Revert import removal

* Fixed mypi issues or added ignores

* Fixed schema formatting

* Fixed hogvm failing tests

* Remove duplicate key in list
2023-10-26 12:38:15 +02:00
Karl-Aksel Puulmann
4ea25abbab
chore(time-to-see-data): Make ClickHouse query tagging thread-safe (#12861)
* chore(time-to-see-data): Make query tagging thread-safe

* Resolve test fail

* Remove excessive commenting

* benchmark helpers
2022-11-21 09:16:22 +02:00
Guido Iaquinti
fa7c706af8
style(flake8): enable E402 (#11707) 2022-09-07 16:11:55 +02:00
Michael Matloka
5d2ad6c7bb
chore(deps): Update black to 22.8.0 (#11596)
* chore(deps): Update `black` to `22.8.0`

* Format
2022-09-05 14:38:54 +02:00
Eric Duong
92ddf4072b
feat: materialize person and groups on events (#11363) 2022-08-30 12:31:32 +01:00
Eric Duong
46faa8fab2
refactor(FOSS): foss remove all ee dependencies from /posthog (#10319)
* refactor: foss move properties to /posthog

* refactor: move replication and materializtaion

* refactor: move file

* refactor: move test and journeys

* refactor: move breakdown props

* refactor: move query imports

* refactor: move more ee dependencies

* refactor: restore groupsjoinquery

* fix: errors

* refactor: no ee.clickhouse dependncies

* try import tasks

* refactor: move materialization

* refactor: change foss split for column optimizer

* run black

* fix: imports

* remove comment

* Update snapshots

* run black

* skip isort

* Update snapshots

* format

* more fixes

* refactor(FOSS): split out paths query code (#10378)

* refactor: move migrations

* refactor: move idl

* fix: move more imports

* import adjustments

* fix: test import

* fix: test import

* fix: remove restriction

* refactor: split out paths query code

* refactor: more code splitting

* fix: types

* refactor(FOSS): Setup such that FOSS is deployable (#10352)

* refactor: move migrations

* refactor: move idl

* fix: move more imports

* import adjustments

* fix: test import

* fix: test import

* fix: remove restriction

* fix import

* refactor: add snapshost

* subscription-import

* fix: safe import

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-22 17:24:03 -04:00
Eric Duong
ce7988ecfa
refactor: move prop value and actorbasequery files (#10258)
* Revert "fix: revert "move prop value and actorbasequery files" due to flaky test (#10257)"

This reverts commit ac1a485cdf.

* chore: empty
2022-06-13 16:24:14 -04:00
Eric Duong
ac1a485cdf
fix: revert "move prop value and actorbasequery files" due to flaky test (#10257)
* Revert "refactor: move prop value and actorbasequery files (#10220)"

This reverts commit 12bf3a7c3e.

* chore: empty
2022-06-10 23:49:53 +00:00
Eric Duong
12bf3a7c3e
refactor: move prop value and actorbasequery files (#10220)
* refactor: move prop value and actorbasequery files

* refactor: move test base

* fix: imports

* move middleware

* refactor: move cohort model util (#10224)
2022-06-10 18:44:50 -04:00
Eric Duong
e5df72b4be
refactor: foss trends (#10206)
* refactor: move files

* refactor: variable naming

* factory pattern

* fix: restore exceptions hog

* remove factory pattern

* refactor: move files and change insight class

* move sql

* types

* fix imports

* fix imports

* refactor: propogate class

* fix import

* refactor: move paths to /posthog

* change imports for clickhousepathsactors

* change order

* fix: restructure imports

* fix imports

* move tests

* add dependency

* remove deprecated

* move tests

* inherit foss tests

* refactor: split stickiness /posthog and /ee (#10085)

* refactor: move files

* refactor: variable naming

* factory pattern

* fix: restore exceptions hog

* remove factory pattern

* fix imports

* fix import

* move test paths

* restore factory pattern

* add mixin

* change imports

* refactor: move funnels and split according to foss

* ignore type on init

* fix: bad import

* refactor: foss trends

* fix: missing import

* fix: missing import

* Update snapshots

* split tests

* move mixin

* move import and snapshot

* fix test

* fix: test

* fix comparison

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-09 17:39:32 +00:00
Eric Duong
9adfaaaba6
refactor: move path query class from /ee to /posthog (#10140)
* refactor: move files

* refactor: variable naming

* factory pattern

* fix: restore exceptions hog

* remove factory pattern

* refactor: move files and change insight class

* move sql

* types

* fix imports

* fix imports

* refactor: propogate class

* fix import

* refactor: move paths to /posthog

* change imports for clickhousepathsactors

* change order

* fix: restructure imports

* fix imports

* move tests

* add dependency

* remove deprecated

* move tests

* inherit foss tests

* refactor: split stickiness /posthog and /ee (#10085)

* refactor: move files

* refactor: variable naming

* factory pattern

* fix: restore exceptions hog

* remove factory pattern

* fix imports

* fix import

* move test paths

* restore factory pattern

* add mixin

* change imports

* refactor: Foss funnels (#10204)

* refactor: move funnels and split according to foss

* ignore type on init

* fix: bad import
2022-06-09 15:09:00 +00:00
Eric Duong
e61b23cbcc
refactor: move retention query class from /ee to /posthog (#10137)
* refactor: move files and change insight class

* move sql

* types

* fix imports

* refactor: propogate class

* change order

* fix: restructure imports

* fix imports

* move tests

* add dependency

* remove deprecated

* inherit foss tests

* refactor: split stickiness /posthog and /ee (#10085)

* refactor: move files

* refactor: variable naming

* factory pattern

* fix: restore exceptions hog

* remove factory pattern

* fix imports

* fix import

* Update snapshots

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-07 12:52:45 -04:00
Eric Duong
5eb98ef624
refactor: move event and person query to /posthog (#8970)
* refactor: move event and person query to /posthog

* move imports

* another import

* refactor: move actions

* refactor: move actions

* enterprise

* trigger tests

* test

* make dummy
2022-03-22 20:27:11 +00:00
Tim Glaser
dc1efb115e
perf: aggregate by distinct id (#8935)
* Perf: aggregate by distinct id

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
2022-03-09 18:17:56 +00:00
Guido Iaquinti
0ef7f6a493
ci(flake8): enable unused import check (#8714) 2022-03-08 09:53:54 +01:00
Marius Andra
07e2489c74
fix(property-filters): materialized property value search (#8718)
* use mat properties for event and person prop value autocomplete

* ignore is_deleted = true person properties

* bugfixes

* remove whitespace

* typo

* test and snapshot materialized columns

* test and snapshot materialized columns

* extract property value query helpers

* add benchmark for property values

* fix pycharm auto-import
2022-03-07 13:02:50 +01:00
Guido Iaquinti
1d1d6d335c
flake8: enable E266,W291,W293,W391,W601 (#8699) 2022-02-21 15:47:27 +01:00
Eric Duong
e4f65b1790
8356 versioning (#8449)
* version insights

* version and lock update

* make sure all tests work

* restore exception

* fix test

* fix test

* add specific id

* update plugin server test utils

* cleanup

* match filtering

* use timestamp comparison

* make tests work

* one more test field

* fix more tests

* more cleanup

* lock frontend when updating and restore refresh

* pass undefined

* add timestamp to background update

* use incrementer

* add field

* snapshot

* types

* more cleanup

* update tests

* remove crumbs

* use expressions

* make nullable

* batch delete

* fill null for static cohorts

* batch_delete

* typing

* remove queryset function
2022-02-09 12:19:00 -05:00
Karl-Aksel Puulmann
d459dc4ca8
Benchmarks: trends breakdown and lifecycle with filtering (#8018)
* Add benchmarks for trends breakdown

* Add lifecycle benchmarks
2022-01-13 11:53:00 +02:00
Karl-Aksel Puulmann
2139a6e204
Add measure.sh helper script to repo (#7877) 2022-01-05 12:46:08 +02:00
Karl-Aksel Puulmann
f3fc669cda
Add benchmark for earliest timestamp sql (#7848) 2021-12-28 15:16:47 +02:00
Karl-Aksel Puulmann
afce8efafb
Add benchmark for funnel query (#7813)
* Add benchmark for funnel query

Testing new sorting order takes the benchmark from 3.5s -> 1.5s \o/

* Update snapshots
2021-12-21 13:09:00 +02:00
Harry Waye
5fb811a365
perf(lifecycle): add benchmark for lifecycle (#7753)
* perf(lifecycle): add benchmark for lifecycle

* Add some notes on running benchmarks locally
2021-12-17 09:36:49 +00:00
Eric Duong
a051f7ee1f
[actors] retention actors (#7495)
* convert to actor form

* change var name

* remove unused imports

* typing issue

* use subquery

* bad import

* groups for general retention query

* actor in period

* update imports

* update test

* remove comment
2021-12-09 10:11:21 -05:00
Karl-Aksel Puulmann
882d26f4f0
Add daily active users benchmarks (#7556) 2021-12-07 18:17:03 +02:00
James Greenhill
b7df925186
Revert "Add logging to all postgresql queries with query context (#7486)" (#7512)
This reverts commit 3287c48f53.
2021-12-03 12:07:19 -08:00
Eric Duong
4e0e69ac91
[actors] Stickiness actor pattern (#7440)
* separate class

* filter typing

* remove repeat

* refactor test

* add limits

* remove ee

* working tests

* working group tests for stickiness

* remove inits

* suggested changes

* typing

* adjust typing

* add correct params

* remove repeat
2021-12-02 11:43:20 -05:00
Karl-Aksel Puulmann
3287c48f53
Add logging to all postgresql queries with query context (#7486)
* Add logging to all postgresql queries with query context

Uses the exact same pattern as we do currently for clickhouse, just
hooking in there differently

* Support psycopg2.sql.SQL

* Better docs

* update a test
2021-12-02 16:08:26 +02:00
Harry Waye
79816a9715
feat(retention): add support for multiple breakdown props (#7431)
* test(retention): add http api tests for retention breakdowns

This just adds tests for person and event property breakdowns

* Add helper function for substituting clickhouse sql params

* feat(retention): add support for multiple breakdown props

This just reuses the work done for funnel multiple breakdown values. I
haven't tested this with anything other than person or event properties.
Rather than try to get it working for all the other property types.

The change adds a new `breakdowns` parameter to the retention endpoint,
that is the same as for funnels, e.g. it looks like:

```
{
    ...
    "breakdowns": [{"type": "person", "property": "os"}, ...]
    ...
}
```

The return structure is the same as the non-breakdown version, except we
also include a `breakdown_values` property that is e.g. `["Chrome",
"95"]`, and the `label` attribute for this case would be "Chrome::95".

* fix typing

* update query snapshots

* remove unused imports

* use `self.assertEqual` instead of `assert`

* Remove explicit should_join_persons

* Revert limit by changes, use breakdown_type

* update snapshots

* fix typing

* use json_encode_request_params in retention test requests

* update json_encode_request_params to encode_get_request_params to better reflect purpose

* perf(retention): add benchmark query for retention with breakdown

* no materialize
2021-12-02 13:35:33 +00:00
Harry Waye
eed62dad9f
perf(retention): add benchmark cases for retention (#7483)
This was pretty much as copy paste job from the stickiness cases. I'm
going to be merging in some changes to retention so I want to get this
in first such that we can see if it affects the perf of these queries at
all. I don't think it touches them, as seen by the snapshot queries
generated by the retention tests not changing in
https://github.com/PostHog/posthog/pull/7431 but better safe than sorry.

Addresses [this comment](https://github.com/PostHog/posthog/pull/7431#pullrequestreview-820773405)
2021-12-02 10:38:49 +00:00
Karl-Aksel Puulmann
98a58648e2
Add benchmarks for session recording page (#7251) 2021-11-22 20:10:09 +00:00
Karl-Aksel Puulmann
bf28d14aed
Add some benchmarks for stickiness (#7116)
* Add benchmarks for stickiness

* Don't require get_earliest_timestamp always

* Improve benchmark
2021-11-15 13:38:11 +02:00
Neil Kakkar
283319adfc
Modify query to not use properties for autocapture (#6711)
* modify query to not use properties for autocapture

* address comments

* add benchmark test to check if worth materialising
2021-10-28 13:12:26 +01:00
Neil Kakkar
850075979c
Add event with properties benchmark (#6666) 2021-10-26 17:19:24 +01:00
Karl-Aksel Puulmann
becaee3abd
Add benchmarks for trends + actions (#6354) 2021-10-12 10:18:23 +03:00
Neil Kakkar
7ff7c5b7cc
Update param name in benchmarks (#6324) 2021-10-08 14:44:00 +03:00
Karl-Aksel Puulmann
7461f90153
Simplify cohort filters (#6277)
* WIP: Create new property types for simplified cohorts

* Add documentation on simplified_cohort_filter_properties

* Handle static-cohort/precalculated-cohort property types

* Handle new property filters properly

* Add casting

* Test cohorts in more cases

* Fix a bug

* Fix benchmark simplifying

* Avoid redoing work every setup for benchmarks

* Update typing;

* Remove unneeded scope

* Add tests for simplifying and cohorts

* Roll more of "do we need to join persons table" behavior into ClickhousePersonQuery class

* Handle precalculated cohort logic in sessions

* Simplify event query

* More tests without any JSONExtract

* Simplify entity properties as well

* Improve docstring

* Add test for breakdown & precalculated cohorts

* Add test for filtering sessions by precalculated cohorts

* Reset unneeded change

* Update cohort

* Solve some typing issues

* Update benchmarking

* Fix cohort filtering tests

* Fix cohort tests

* Fix a caching issue

* Typecheck

* Handle exclusion filters
2021-10-08 10:51:11 +03:00
Neil Kakkar
cfe9844a09
Add benchmarks for correlation (#6307)
* Add benchmarks for correlation
2021-10-07 14:12:56 +00:00
Karl-Aksel Puulmann
fda8313be6
Use fixed benchmarking version (#6247)
This way adding new benchmarks/setup code does not invalidate the
existing code.
2021-10-05 13:24:10 +03:00
Karl-Aksel Puulmann
0512972426
Add benchmarks for cohort filtering in trends (#6244)
* Add benchmarks for cohort filtering in trends

* Add a way to manually trigger benchmarking
2021-10-05 12:16:55 +03:00
Karl-Aksel Puulmann
437b2982cd
Benchmarking suite for ClickHouse queries (#6187)
* Add asv code

* Get a benchmark running

* better output folder

* Move benchmark file

* Cleanup of config

* Run a query and benchmark

* Skip benchmarks for isort

* Set up materialized columns before test

* Better skipping logic

* Add first proper benchmark, add some documentation

* Person property filtering

* Add new workflow

* Show stderr in test run

* Continue debugging actions

* Try to save benchmark results to separate repo

* Output to main directory

* Scheduling

* Remove if temporarily

* Int for ms

* Comment on the PR

* Collapsible section

* Update README

* Clarification

* Remove h3

* Remove pyproject.toml, .gitignore changes that are unneeded
2021-10-01 15:20:58 +03:00