0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-24 18:07:17 +01:00
Commit Graph

102 Commits

Author SHA1 Message Date
Paul D'Ambra
50b12d9bb7
Set debug, test, and primary_db env variables based on context (#7250)
* set debug, test, and primary_db env variables based on context

* remove unnecessary print

* also use pytest.ini for some tests run straight from pytest at the CLI
2021-11-20 17:18:39 +00:00
Yakko Majuri
963c62aaf1
Use monorepo plugin server (#6784)
* use plugin-server in same repo

* remove plugins dir

* update docker files

* clean up docker files:

* visual separation

* run plugin server test stack

* build and push test image

* revert push image

* fix default yarn start

* another fix

* fix some more

* add plugin-server-prod script

* make executable

* update scripts:

* revert prod script

* fixes

* fix tests

* clean

* move build to docker

* build plugin-server in prod dockerfile

* use different entrypoint for e2e test

* fix bugs with ts-node-dev

* restore yarn.lock

* lost a few changes in the rebase

* fix dockerfile

Co-authored-by: James Greenhill <fuziontech@gmail.com>
2021-11-13 11:52:00 -08:00
Yakko Majuri
79744990e2
fix issues, logging (#6998)
🚢
2021-11-09 09:05:22 -08:00
Paul D'Ambra
0cdfb2531a
Only load events table from the URL on allow list of URLs (#6949)
* only load events table from the URL on allow list of URLs

* add return type to function

* pass scene URL into events table

* use urlpattern to check if scene is loaded in events table

* don't need to check if still on the same page as when loaded because urltoaction is now specific

* which means we don't need url pattern matching at all

* add scene url to events scene export's logic props

* use a different action name for each run or retries can never complete as the action cannot be saved

* do not poll results if moved away from sceneUrl

* remove tabs from person/action event tables

* so cypress can run locally

* break the dependency of running one test first so the other can pass

* move check into logic and add an e2e test

Co-authored-by: Marius Andra <marius.andra@gmail.com>
2021-11-09 15:35:41 +00:00
Karl-Aksel Puulmann
fa79f8ea67
BE (Groups/Trends): Allow aggregating by groups (#6894)
* Type math in Entity

* Allow passing group_type_index from FE to BE

* Get a initial query running

* Add group value filter if aggregating by groups

* Add snapshot testing for trends queries

* isort

* Update tests

* Add test for column_optimizer

* Update ee/clickhouse/queries/trends/util.py

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>

Co-authored-by: Neil Kakkar <neilkakkar@gmail.com>
2021-11-05 13:08:32 +02:00
Yakko Majuri
6ce204fe3c
add version to person (#6628)
* add version to person

* add migration

* set all initial version values to 0

* remove defaults

* fix dep
2021-10-28 15:43:30 +00:00
Paul D'Ambra
f2338c7674
you always need DEBUG=1 when running tests (#6613) 2021-10-22 13:37:01 +00:00
Paolo D'Amico
20b0f9704a
Forgot password UI (#6177) 2021-10-15 10:40:31 +03:00
Michael Matloka
77621366de
Fix remaining TypeScript errors (#6436)
* Fix remaining TS errors and remove `.ts-strict-blacklist`

* Move `IGNORED_SPECS` to top
2021-10-14 15:08:42 +00:00
Yakko Majuri
e5a9c3b04e
merge people test (#5567)
* do all merge_people ops in plugin server

* add integration test

* flip call order

* allow calls in any order

* update test approach
2021-08-16 10:53:52 +03:00
Yakko Majuri
3b8d328c10
Allow passing in token to Personal API Key requests (#5044)
* Allow passing in token to Personal API Key requests

* fix

* fix accidental commit

* update comment

* dont access self.request.user.team if not needed

* add tests

* Add an assert to make mypy happy

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-07-16 10:35:56 -03:00
Karl-Aksel Puulmann
bbd41ef04a
Use -Ofair for celery workers (#5063)
From https://medium.com/@taylorhughes/three-quick-tips-from-two-years-with-celery-c05ff9d7f9eb

> By default, preforking Celery workers distribute tasks to their worker processes as soon as they are received, regardless of whether the process is currently busy with other tasks.
> If you have a set of tasks that take varying amounts of time to complete — either deliberately or due to unpredictable network conditions, etc. — this will cause unexpected delays in total execution time for tasks in the queue.

This is 100% the case for us. This should "load balance" the tasks
better across workers.
2021-07-12 13:52:05 +03:00
Paolo D'Amico
01b7daacc2
Remove NewRelic from Cloud (#4716)
* remove NR

* remove secret key too
2021-06-14 11:01:57 +02:00
Tim Glaser
bbb4025ab1
Fix e2e tests (#4357)
* Fix e2e tests

* Run as 'production' to better mimick CI

* try no self capture

* Correct event/property definitions

* fix

* fix system status

* Try caching cypress separately

* Cache node modules

* cache pip better

* fix tests

* dont run if not cache

* test

* Cache cypress

* Fix event definitions

* Fix and speed up

* try fixing

* fix

* fix

* faster install

* try self capture 1

* fix multiple

* fix

* fix

* fix

* more speeding up

* Fix tests

* try not installing deps

* fix test global

* possibly quicker

* correct

* Fix
2021-05-20 16:42:26 +02:00
Sam Winslow
ca1a7bfc4e
Add bin script to start Cypress component tests (#4186)
* Add bin script to start cypress component test

* remove unneeded flags
2021-05-04 17:14:17 -04:00
Tim Glaser
3520637967
Warn before running all tests (#4204) 2021-05-04 15:41:22 +02:00
Buddy Williams
c893765973
Docker support on the Apple M1 chip / arm64 (#4024)
* First pass at docker support on the Apple M1 chip

* Added version to libpq-dev so that it passes linter checks; Removed comments
2021-04-19 10:12:02 -04:00
Paolo D'Amico
2020c32c66
Add NewRelic to cloud deployment (#4005) 2021-04-15 11:53:19 -07:00
Sam Winslow
49f7d841ce
bump cryptography==3.4.7 and add macosx_arm64 install script (#3935) 2021-04-10 00:53:19 +02:00
Eric Duong
5b77823938
Add cypress dependency installation (#3914)
Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2021-04-08 10:14:06 -07:00
Tim Glaser
1918675f93
Default set debug=1 to bin/start (#3807)
* Default set debug=1 to bin/start

* Export DEBUG

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-04-06 15:50:48 +02:00
Michael Matloka
446094d3dd
Get rid of DATABASE_URL and REDIS_URL construction in bin/plugin-server (#3873) 2021-04-06 11:39:30 +02:00
Michael Matloka
525e09c358
Delete docker-preview (#3842) 2021-04-01 16:46:09 +02:00
Paolo D'Amico
89b71a9959
New login & signup (React) (#3701) 2021-03-25 07:40:20 -07:00
Marius Andra
f1d0a5f1ca
remove ioredis debug (#3568) 2021-03-03 14:14:16 +01:00
Tim Glaser
c153753d04
Fix bin/tests (#3540) 2021-03-01 20:34:39 +01:00
Karl-Aksel Puulmann
152f799351
e2e-test-runner: Only start webpack if not already running (#3484) 2021-02-26 16:50:21 +02:00
Karl-Aksel Puulmann
24de8b55e4
Run tests via pytest (#3417)
* Compile requirements-dev.txt with latest pip-tools

* Install pytest

* Avoid picking up factories as tests

* New runner

* Always set TEST env variable running tests

Some of our tests rely on it.

* Remove repetition

* Fix a broken test

* Cut down noise from bin/tests

* Rename test factory

* Fix stickiness filter

* Skip a broken test

This has been broken since numpy removal PR. Sadly tests were not
running for this submodule

* Fix import on ee

* Run ee tests properly

The django_db_setup fixture will be automatically run when running ee/
module tests.

* Make tests run on CI

* Include REDIS_URL, fix cloud

* Set TEST env variable

* Hack cloud tests to work

* Attempt at workflow fix

* Import Person model when running ee tests

This module implicitly adds hooks, so this is needed when running tests

* Respect reuse-db for clickhouse

* Add custom markers to avoid warnings

* pytest: use ch test database always

Accidentally wiped by ch setup a few times without this. Oops

* Remove repetition in tests

* Pytest: Always run migrations

Testing a state cleanup fix

* Use same DB in conftest and main code

* Pytest: autoset TEST setting without env variable

* fix broken test

Co-authored-by: eric <eeoneric@gmail.com>
2021-02-24 09:32:44 +02:00
Marius Andra
1601740bda
add : in front of password for ioredis (#3439) 2021-02-23 13:04:33 +01:00
Marius Andra
817e117da1
Send 20% of events to plugin server (#3437)
* send 20% of events to plugin server

* add ioredis debug
2021-02-23 10:26:26 +01:00
Michael Matloka
edbc547d73
Support POSTHOG_REDIS_* vars in bin/plugin-server (#3426) 2021-02-22 19:15:45 +01:00
Karl-Aksel Puulmann
4ad6a28e60
Auto log in for e2e tests and review apps (#3377)
* Add AUTO_LOG_IN environment variable

This will be used when running tests on CI, setting up demo instances
and with cypress vcr

* Auto log in in review apps, e2e tests

This will speed up tests/reviewing significantly

* Rename AUTO_LOG_IN to AUTO_LOGIN

Co-authored-by: Michael Matloka <dev@twixes.com>
2021-02-18 13:59:09 +01:00
Karl-Aksel Puulmann
30b6692920
React component tests with cypress (#3233)
* WIP: get cypress something doing

* Get a test running

* Get css loading, stub posthog

* Move helpers to separate file

* Givens > given2

It plays nicer with beforeEach

* Test for session filters

* Test date navigation

* Test filtering

* Try keep old tests running

* Try get tests running under CI

* Prettify fixtures

* Cleanup, use cypress 6

* Add yarn build

* Fix e2e cypress tests

* given2 => givens

* Rename `frontend-test-runner` to `e2e-test-runner`

* Fix cypress test

* Add webpack-preprocessor as a devDependency

* Improve freezing time

* Make css inclusions automatic for component tests
2021-02-09 17:28:01 +02:00
Karl-Aksel Puulmann
a978b28a68
Run typescript with strict in CI, allow no new errors (#3050)
* Verify no new typescript code breaks "strict" compilation

With this we have a file containing all current errors. As the code
changes, we should not add more errors here.

* remove some lines to show a failure

* Add total count to script

* Update blacklist after rebase

* Remove some fails on purpose.

* Attempt a reorder

* Add missing file

* Update script to write typegen

* Refresh list

* Attempt to use normal tsc

* update blacklist

* Store all errors

* echo for typegen

* Add check command

* typegen 3 times

* typegen 4 times

* typegen 6 times

* typegen 10 times

* remove debug code

* Update kea typegen to latest

* Remove hacks
2021-01-22 16:36:30 +02:00
Michael Matloka
eaa169100a
Add handing off event ingestion to plugin server (#2898)
* Add setting for handing off process_event_ee to plugin server

* Add StatsD settings to KEYS

* bin/plugin-server → start-plugin-server & docker-plugin-server

* Simplify to only add docker-plugin-server

* Bring back original comment

* Turn down verbosity of plugin server install

* Remove redundant if

* Fix comment

* Remove lone newline

* Roll back unsafe script changes

* Simplify dockerized plugins

* Add some depends_on

* Clarify HAND_OFF_INGESTION env var

* Use posthog-plugin-server 1.0.0-alpha.1

* Enhance bin/plugin-server and rm bin/docker-plugin-server

* Move around PLUGIN_SERVER_INGESTION_HANDOFF ifs

* Use posthog-plugin-server@1.0.0-alpha.2

* Support kafka+ssl:// in plugin-server

* Produce to topic events_ingestion_handoff for plugin server

* Use posthog-plugin-server@1.0.0-alpha.3

* Don't import Kafka topics in FOSS

* Use @posthog/plugin-server

* Update yarn.lock

* Add commands for external ClickHouse setup/teardown

* Actually delete test CH teardown command

* ClickhouseTestRunner.setup_test_environment() in setup_test_clickhouse

* Rework test setup script to work with Postgres too

* Restore master plugins dir for merge

* Unset PLUGIN_SERVER_INGESTION_HANDOFF in docker-compose.ch.yml

* Fix unimportant typo

* Build log_event data dict only once

* Make it clear in bin/plugin-server help that it's bin

* Space space
2021-01-21 15:39:44 +01:00
Marius Andra
39239c2780
Upgrade Plugin Server (#2816)
* use KAFKA_ENABLED key

* 0.6.1 plugin server
2020-12-21 21:57:29 +01:00
James Greenhill
3d0413bdcb
Verbose Yarn build on plugin server build (#2805) 2020-12-16 18:06:56 -08:00
James Greenhill
d93ae873a6
Make building with yarn more verbose for debugging (#2804) 2020-12-16 17:14:57 -08:00
James Greenhill
a8de2ac587
plugins debug (#2803)
* checkin

* Don't run yarn silently for plugin-server
2020-12-16 16:35:19 -08:00
Marius Andra
c191fef86b
Ignore plugins optional deps (#2801)
* ignore optional deps when installing plugin-server

* --ignore-optional in docker
2020-12-16 23:59:40 +01:00
Michael Matloka
7ba9f7de09
Plugin server ingestion base (#2732)
* Add relevant settings to KEYS in bin/plugins-server

* Log all EE events to events_handoff Kafka topic for plugin server

* Clean up settings

* Fix FOSS

* Don't introduce KAFKA_EVENTS_HANDOFF

* Add cosmetic newline

* Add DEBUG WAL print()
2020-12-14 16:05:18 +01:00
Tim Glaser
560fe0765e
Deprecate dashboard item type and move to display (#2715)
* Deprecate dashboard item type and move to display

* Mypy rerors

* fix test

* fix

* Fix test

* Fix another test

* Remove determineInsightType

* Get rid of RetentionTable display types

* Fix update dashboard
2020-12-11 20:12:52 +01:00
Marius Andra
b5b7276ced
Restart plugins on error (#2719)
* export as $CONFIG_JSON

* export as $CONFIG instead

* restart plugin server if killed

* restart plugin server if killed

* update to 0.4.3
2020-12-10 07:29:06 +01:00
Marius Andra
177cee9e38
Set right concurrency for plugin server on Heroku (#2703)
* nodejs worker concurrency on heroku

* upgrade plugin server

* upgrade server

* fix another bug with 0.3.2

* plugin server with resilient redis

* plugin server with more resilient workers

* new plugin server

* update
2020-12-09 18:08:25 +01:00
Michael Matloka
b39dce42cf
Fix webpack-dev-server inside Docker (#2699) 2020-12-08 11:44:39 +01:00
Marius Andra
c24aebe174
Add flags to bin/docker-worker-celery (#2621)
* add flags to docker-worker-celery and rename --with-beat to --with-scheduler

* Clean script up

Co-authored-by: Michael Matloka <dev@twixes.com>
2020-12-03 18:53:06 +01:00
James Greenhill
908cfaf2c8
Don't run preview docker build with debug flags (#2610)
* Don't run web in debug for preview docker build

* worker to not debug
2020-12-01 15:15:30 -08:00
Karl-Aksel Puulmann
df23e3353d
Fix /bin/tests (#2527)
/bin/tests did not work for me anymore - I think this was the original
intent?
2020-11-26 15:32:38 +01:00
Marius Andra
b8d1ba43f4
Disable celery heartbeat, gossip and mingle (#2513)
* disable celery heartbeat, gossip and mingle

* keep the heartbeat

* beat doesn't want these params

* add back "without heartbeat"
2020-11-26 12:44:10 +01:00
Marius Andra
eb8b650aee
Docker nodejs support for plugins (#2452)
* keep npm around in the dockerfiles (needed for plugin-server)

* simplify docker-preview script
2020-11-20 17:04:10 +01:00