0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-12-01 12:21:02 +01:00
Commit Graph

124 Commits

Author SHA1 Message Date
Karl-Aksel Puulmann
40a1721d0b
fix: Simplify test setup for plugin-server (#9051)
We don't need multiple commands for setup, also verify TEST env is set.
2022-03-17 10:34:53 +02:00
Karl-Aksel Puulmann
c3649f91c9
feat(sharding): Remove CLICKHOUSE_REPLICATION (#9013)
* Check async migrations instead of CLICKHOUSE_REPLICATION for mat columns

* Update a comment

* Default for CLICKHOUSE_REPLICATION

* add replication file

* Assert is replicated in tests

* Remove DROP TABLE query from cohortpeople migration

* Update snapshots

* Ignore migration in typechecker

* Truncate right table

* Add KAFKA_COLUMNS to distributed tables
2022-03-16 10:27:39 +02:00
Harry Waye
b67ec6f9f8
chore(plugin-server): update babelrc node version presets (#9032)
* chore(plugin-server): update babelrc node version presets

While updating node versions in some places, we missed this one.

* bump version in devcontainer for plugin-server
2022-03-15 14:41:30 +00:00
Yakko Majuri
3ab69d23e3
chore(deps): upgrade node.js version to v16 (#8983)
* chore(deps): upgrade node.js version to v16

* update npm version'

* try to update alpine

* Revert "try to update alpine"

This reverts commit 58024968e2.

* don't update alpine, use node-current instead:

* go back to npm 7

* upgrade @google-cloud packages

* handle unhandled rejections :D

* add handling for unhandled promise rejections in the main thread as well
2022-03-15 09:48:56 +00:00
James Greenhill
2ff06af65f
fix: Plugin server webhook token extractor regex ignores '$' characters in key (#9023) 2022-03-14 17:13:55 -07:00
Yakko Majuri
b26241c614
Revert "fix(deploy-race-condition): retrigger deploy (#9006)" (#9011)
This reverts commit aa5f741da5.
2022-03-14 12:54:00 +00:00
Yakko Majuri
aa5f741da5
fix(deploy-race-condition): retrigger deploy (#9006) 2022-03-14 12:01:44 +00:00
Yakko Majuri
25dd9ce843
fix(plugin-server): bring back historical export with safety mechanisms (#9003) 2022-03-14 10:53:18 +00:00
Yakko Majuri
814090b905
perf(plugin-server): disable exportHistoricalEvents on cloud (#8997) 2022-03-12 09:10:41 +00:00
Yakko Majuri
ba19eba56d
refactor(plugin-server-logs): show thread info before log (#8966) 2022-03-10 16:52:01 +00:00
Yakko Majuri
18860fc49f
feat(plugin-server): add structured logging (#8888)
* feat(plugin-server): add structured logging

* updates

* handle methods

* fix

* use structlog

* fix deps
2022-03-10 12:58:06 +00:00
Yakko Majuri
77a3790abf
chore(plugin-server-tests): refactor tests to use a top-level describe (#8955) 2022-03-10 11:18:24 +00:00
dependabot[bot]
f27bbf6ad0
chore(plugin-server): bump vm2 from 3.9.5 to 3.9.6 in /plugin-server (#8594)
Bumps [vm2](https://github.com/patriksimek/vm2) from 3.9.5 to 3.9.6.
- [Release notes](https://github.com/patriksimek/vm2/releases)
- [Changelog](https://github.com/patriksimek/vm2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/patriksimek/vm2/compare/3.9.5...3.9.6)

---
updated-dependencies:
- dependency-name: vm2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-10 09:58:10 +00:00
Yakko Majuri
235c263016
perf(plugins): setupPlugin retries refactor (#8900)
* refactor(plugins): accept pluginConfig and hub in lazyvm constructor

* perf(plugins): refactor setupPlugin retries mechanism

* remove log

* fix test as clearError is now on the setupPlugin path

* fix?

* test
2022-03-09 15:26:34 +00:00
Yakko Majuri
01f6d4dcdd
feat(ingestion): add metric to eachBatch failures (#8884)
* feat(ingestion): add metric to eachBatch failures

* add increment metric for runInstrumentedFunction
2022-03-07 13:24:54 +00:00
Yakko Majuri
774b16d376
fix(plugin-server): immediate mitigation for setupPlugin issues (#8820)
* reduce frequency and total number of setupPlugin retries

* add delay

* fix test

* further test fixes

* better check

* fixes

* fixes
2022-03-04 13:57:18 +00:00
Yakko Majuri
50f128aadb
refactor(plugin-server): remove status report (#8778)
* remove captureTimeSpentRunning

* remove status report altogether
2022-03-04 13:18:54 +00:00
Yakko Majuri
2aab6eb89b
refactor(plugins): accept pluginConfig and hub in lazyvm constructor (#8843)
* refactor(plugins): accept pluginConfig and hub in lazyvm constructor

* fix

* fix test
2022-03-04 13:18:35 +00:00
Yakko Majuri
2af5df0f29
feat(ingestion): add metric for plugin server time to ingestion-ready (#8858) 2022-03-04 11:45:34 +00:00
Yakko Majuri
404fe90ffd
plugin server readiness endpoint (#8819) 2022-03-03 12:35:38 +00:00
Yakko Majuri
240ed011e5
add metric on posthog.capture inside plugins (#8810) 2022-03-02 14:14:53 +00:00
Yakko Majuri
5416d78fcd
add one more ingestion backpressure metric for comparison (#8784) 2022-03-01 14:59:14 +00:00
Yakko Majuri
67ba234237
add ingestion metric using increment (#8779) 2022-03-01 11:47:10 +00:00
Tim Glaser
90d30c3e60
Plugins: replace @current with team_id in api calls (#8751)
* Plugins: replace @current with team_id in api calls

* prettier

* Fix test

* reallyf ix test
2022-02-28 17:29:01 +00:00
Yakko Majuri
378809998a
track when a plugin drops an event so we don't ingest it (#8767)
* track when a plugin drops an event so we don't ingest it

* Update plugin-server/src/main/ingestion-queues/ingest-event.ts

Co-authored-by: Tiina Turban <tiina303@gmail.com>

Co-authored-by: Tiina Turban <tiina303@gmail.com>
2022-02-28 17:21:15 +00:00
Paolo D'Amico
9496394952
Remove & clean onboarding-2822 (#8707) 2022-02-23 08:22:01 +01:00
Yakko Majuri
aef30d9a41
emit piscina task to reloadSchedule on reload call (#8713) 2022-02-22 15:25:00 +01:00
Yakko Majuri
64eef6d0a1
avoid sending another reloadSchedule task on plugin worker setup (#8694)
* avoid sending another reloadSchedule task on plugin worker setup

* update tests
2022-02-22 13:03:28 +01:00
Yakko Majuri
571c902a7a
Super Lazy VMs (#8609)
* super lazy vms

* address review

* update tests

* fix typo

* test updates

* ts magic

* sort stuff out

* don't import from another test

* fix lazy tests and hopefully everything will be right in the world

* update teardown tests

* increase timeout in e2e.timeout

* sort imports

* test delay

* import delay

* fix leaky failures and 🚀
2022-02-18 11:52:38 +00:00
Yakko Majuri
2fc7306571
reduce maximum retries for exportEvents (#8526)
* reduce maximum retries for exportEvents

* Update plugin-server/src/worker/vm/upgrades/export-events.ts

* fix test

* fix
2022-02-17 13:45:33 +00:00
Yakko Majuri
60586bcf16
track when the promise manager awaits a promise (#8672) 2022-02-17 13:39:17 +00:00
Yakko Majuri
b8b69b8f41
refactor update public jobs query (#8596)
* refactor update public jobs query

* void promise

* additional optimization

* fix typo

* Update plugin-server/src/utils/db/db.ts

Co-authored-by: Tiina Turban <tiina303@gmail.com>

Co-authored-by: Tiina Turban <tiina303@gmail.com>
2022-02-16 17:00:20 +00:00
Yakko Majuri
a919c3897a
increase isNewPerson TTL to 4 hours (#8637) 2022-02-16 15:09:51 +00:00
Yakko Majuri
c7810cbab1
Refactor exportEvents buffer (#8573)
* redo export events buffer

* fix implementation

* don't return promise from trackPromise

* Update plugin-server/src/worker/vm/promise-manager.ts

* Promise.any -> Promise.race

* fix test?

* try to increase timeout?

* add a stupid assertion to silence TypeError
2022-02-16 15:07:05 +00:00
Yakko Majuri
89b5bd36ec
Allow disabling a plugin's logs (#8519)
* allow disabling plugin logs

* fix

* suggested log level approach

* fixes
2022-02-16 11:35:26 +00:00
Yakko Majuri
dae78d71c3
Plugin server turbo mode (#8578)
* Plugin server turbo mode

* update type

* also set updated_at on install

* don't hit pubsub if not necessary

* fix migration deps
2022-02-14 17:53:04 +00:00
Yakko Majuri
07f5f565ab
fix api key bot (#8566)
* fix api key bot

* add explicit assurance of hitting cache

* remove log

* revert debug

* prettier

* rename test
2022-02-14 11:55:35 +00:00
Tiina Turban
8eb94fdd37
Disable new person props tests (#8549) 2022-02-11 14:05:34 +01:00
Yakko Majuri
7503d2429e
remove status report hourly (#8534) 2022-02-10 17:02:09 +00:00
Yakko Majuri
d2d6769fd7
more and more plugin server metrics (#8544) 2022-02-10 16:02:47 +00:00
Yakko Majuri
7e74e736f6
fix metrics jobs tags (#8541) 2022-02-10 13:22:45 +00:00
Yakko Majuri
272dab8528
add additional plugin server metrics (#8536) 2022-02-10 11:42:18 +00:00
Eric Duong
e4f65b1790
8356 versioning (#8449)
* version insights

* version and lock update

* make sure all tests work

* restore exception

* fix test

* fix test

* add specific id

* update plugin server test utils

* cleanup

* match filtering

* use timestamp comparison

* make tests work

* one more test field

* fix more tests

* more cleanup

* lock frontend when updating and restore refresh

* pass undefined

* add timestamp to background update

* use incrementer

* add field

* snapshot

* types

* more cleanup

* update tests

* remove crumbs

* use expressions

* make nullable

* batch delete

* fill null for static cohorts

* batch_delete

* typing

* remove queryset function
2022-02-09 12:19:00 -05:00
Paul D'Ambra
5cebefdf37
Improve detection of numbers and add boolean detection (#8490)
* improve detection of numbers and add boolean detection

* remove all detected string property types

* don't mix function declaration styles in one file

* fix some surprising test values

* add stack overflow link for magic

* remove boolean detection

* remove migration

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2022-02-08 19:58:37 +00:00
Yakko Majuri
180b90a98e
add another measure of event loop lag to plugin server (#8495)
* add another measure of event loop lag to plugin server

* add tags
2022-02-08 18:11:36 +00:00
Yakko Majuri
47ef723fa5
track buffer voided promises in plugin server (#8492) 2022-02-08 17:24:51 +00:00
James Greenhill
3d312ad1c6
Use tagging a bit better across the board in plugins (#8475)
* Use tagging a bit better across the board in plugins

* remove specific stats name
2022-02-08 02:12:59 +00:00
James Greenhill
abc1de3972
Add teamId tag for plugin metrics (#8474)
* Add teamId tag for plugin metrics

* handle optional better

* Catch errors on topic creation if topic exists
2022-02-07 17:26:09 -08:00
Yakko Majuri
fca0a46772
Add tags to dead letter queue table (#8345)
* add tags to dead letter queue table

* Update ee/clickhouse/migrations/0023_dead_letter_queue_tags.py

* fix typo

* fixes

* update snapshots

* fix tests

* fix test

* update snapshot

* send tags

* update

* Update ee/clickhouse/migrations/0023_dead_letter_queue_tags.py

* fix plugin server test?

* Update plugin-server/src/worker/ingestion/utils.ts

* Update plugin-server/src/types.ts

* Update plugin-server/src/worker/ingestion/utils.ts
2022-02-04 15:36:15 -05:00
Alex Gyujin Kim
c12179b5ba
Give actions descriptions (#8272)
* add description to action

* update tests

* no limit to description and use pageheader

* add ee_action

* add description as global relationship:

* fix tests

* revert to using descriptions

* revert to using descriptions

* allow description to be null

* fix tests

* persist mode

* fix e2e

* Address review points, clean up some margins

* Clean up the way editing state is determined

* Restore default description value and improve typing

* Fix typing better

* Update sql.ts

Co-authored-by: Michael Matloka <dev@twixes.com>
2022-02-02 15:50:28 +01:00