0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-28 18:26:15 +01:00
Commit Graph

11 Commits

Author SHA1 Message Date
James Greenhill
dedf5582c1
first chunk of clickhouse framework (#1613)
* first chunk of clickhouse framework

* prod, not dev docker-compose.yml

* add clickhouse sql files
2020-09-08 16:12:27 -07:00
James Greenhill
c2e03a3a46
Revert this to bring back the working copy of ee / clickhouse (#1588) 2020-09-04 21:12:07 -07:00
James Greenhill
8631ee232b
Noop on celery worker if ee is not enabled (#1587)
* Noop on celery side if ee is not enabled

* Formatting and stub async and sync executors
2020-09-04 19:35:17 -07:00
James Greenhill
1e905e2913
Update async events to clickhouse to use python 3.6 api (#1586) 2020-09-04 18:37:38 -07:00
James Greenhill
8e26357770
Async writes to clickhouse (#1585)
* Initial commit async-ing clickhouse

* Async writes to clickhouse

* deconflict master

* sync events listing

* mypy checks
2020-09-04 17:46:02 -07:00
James Greenhill
5506135c3c
Bring back clickhouse changes along with queue size metrics (#1579)
* Revert "Revert "Clickhouse setup (#1463)""

This reverts commit 7f2cab4b93.

* add queue backlog to _stats endpoint

* celery queue length into heartbeat -> statsd

* reformat

* type check fixes

* bump expected number of queries to 8

Co-authored-by: Eric <eeoneric@gmail.com>
2020-09-04 13:44:53 -07:00
James Greenhill
7f2cab4b93 Revert "Clickhouse setup (#1463)"
This reverts commit a0327587cb.
Time to process events shot way up and logs are missing.
2020-09-03 19:27:02 -07:00
Eric Duong
a0327587cb
Clickhouse setup (#1463)
* initial

* migration command

* migrations working

* add modelless views for clickhouse

* initial testing structure

* use test factory

* scaffold for all tests

* add insight and person api

* add basic readme

* add client

* change how migrations are run

* add base tables

* ingesting events

* restore delay

* remove print

* updated testing flow

* changed sessions tests

* update tests

* reorganized sql

* parametrize strings

* element list query

* change to seralizer

* add values endpoint

* retrieve with filter

* pruned code to prepare for staged merge

* working ingestion again

* tests for ee

* undo unneeded tests right now

* fix linting

* more typing errors

* fix tests

* add clickhouse image to workflow

* move to right job

* remove django_clickhouse

* return database url

* run super

* remove keepdb

* reordered calls

* fix type

* fractional seconds

* fix type error

* add checks

* remove retention sql

* fix tests

* add property storage and tests

* merge master

* fix tests

* fix tests

* .

* remove keepdb

* format python files

* update CI env vars

* Override defaults and insecure tests

* Update how ClickHouse database gets evaluated

* remove bootstrapping clickhouse database routine

* Don't initialize the clickhouse connection unless we say it's primary

* .

* fixed id generation

* remove dump

* black settings

* empty client

* add param

* move docker-compose for ch to ee dir

* Add _public_ key to repo for verifying self signed cert on server

* update ee compose file for ee dir

* fix a few issues with tls in migrations

* update migrations to be flexible about storage profile and engine

* black settings

* add elements prop tables

Co-authored-by: James Greenhill <jams@uber.com>
2020-09-03 10:27:45 -07:00
Tim Glaser
764bdbd0ff
Fix license issues (#1511)
* Fix license issues

* fix(license): fix license tests

Co-authored-by: Paolo D'Amico <paolodamico@users.noreply.github.com>
2020-08-26 15:53:24 +02:00
Michael Matloka
c040601f49
Personal API keys and Zapier integration (#1281)
* Add missing migration

* Add generate_random_token() model util

* Move PublicTokenAuthentication to utils

* Make use of generate_random_token

* Add User.personal_access_token field

* Add PersonalAccessTokenAuthentication

* Fix PublicTokenAuthentication

* Fix migration and auth import

* Add personal_access_token to user API

* Update Setup.js

* Support trailing slash in API

* Improve PAT auth quality

* Add django-rest-hooks requirement

* Update settings.py for rest_hooks

* Fix django-rest-hooks requirement

* Bring back API routes with no double trailing slash

* Rename posthog.api.team to team_user

* Add API TODO

* Ad PAT auth with X-PAT HTTP header

* Replace User.personal_access_token with PersonalAPIKey model

* Fix PersonalAPIKey max_lengths

* Describe posthog.models.utils.generate_random_token better

* Add personal_api_key to API

* Add authenticate_header to PersonalAPIKeyAuthentication

* Add hook API endpoint

* Use django.utils.timezone in place of datetime.datetime

* Add Personal API Keys to Setup

* Sort personal_api_keys in ORM

* Add Action.on_perform()

* Remove requirements.txt comment

* Add a

* Add REST hook tasks

* Optimize PersonalAPIKeyAuthentication query

* Add a trailing slash version of /e endpoint

* Add team field to PersonalAPIKey model

* Add personal API key support to capture endpoint, get_cached_from_token

* Reject personal API keys from inactive users

* Add extra_properties_json field to /capture

* Improve PAK auth header regex

* Use custom hook model

* Deliver hooks

* Handle action.on_perform

* Consolidate userLogic in userLogic.tsx

* Update PersonalAPIKeys.js

* Make PersonalAPIKey foreign keys read-only

* Update requirements/dev.txt

* Make PersonalAPIKeys TSX

* Fix conflict

* Fix migration

* Fix minor mishaps

* Update and fix tests

* Use CharField of random 32 bits as hook.id

* Fix conflicting migrations

* Fix ValidationError in HookSerializer.validate_event

* Use query param in /api/event/actions ID filtering

* Rename endpoint `hook` to `hooks`

* Satisfy mypy

* Add tests

* Use DRF serialization in action_defined and annotation_created triggers

* Update migration leafs

* Make mypy ignore rest_hooks

* Update Django signal receiver names

* Update TS dependencies

* Revert "Update TS dependencies"

This reverts commit 7fc26fefcd.

* Add field user to Hook model

* Update migration leafs

* Fix circular import

* Fix some code

* Install git before running pip install in Dockerfiles

* Improve personal API keys UI

* Satisfy mypy

* Reword key label placeholder

* Add personal API key support to /api/user/*

Unfortunately these endpoints are still limited by CSRF protections at the moment, so not accessible outside PostHog itself.

* Improve PersonalAPIKeyAuthentication and add CsrfOrKeyViewMiddleware

* Run collectstatic before test

* Don't install dev dependencies in CI

* Update dependency installation order in CI

* Fix bug and describe PersonalAPIKeyAuthentication

* Fix CI issues

* Fix typing issues

* Fix more typing issues

* Use /api/personal_api_keys to list keys

* Move REST hooks (and therefore Zapier) to ee/

* Refactor personal API logic with kea-loaders

* Add "More about API authentication in PostHog docs."

* Update PersonalAPIKeys.tsx

* Use TestMixin

* Fix "Authentication" that should've been "Authorization"

* Add option to skip self.client.force_login in API tests

* Include team_id and user_id in personal API key serialization

* Update test_hooks.py

* Add personal API key tests

* Remove leftover

* Make ee.settings override posthog.settings

* Don't directly import from models

* Remove unused imports

* Fix mypy issues

* Fix HOOK_DELIVERER

* Use decorator for /api/user PAK auth

* Don't fire REST hook if user doesn't have "zapier" feature

* Import Optional

* Reword to "premium Zapier"

* Make mypy happy

* Fix test_delete_personal_api_key

* Fix misclick

* Fix and test /capture with personal API key

* Make mypy happy

* Remove extra_properties_json

* Resolve migrations

* Remove apt-utils

* Optimize and test PAK user.is_active filtering

* Replace DEBUG true with 1

* Remove unused instance_id

* Improve typing

* Fix deletion toast

* Refactor CopyToClipboard and use it in PAKs

* Use toast.success

* Update migrations

* Fix migration

* Fix migrations

* Complete merge

Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-08-26 10:34:57 +02:00
Tim Glaser
23f5918477
Add enterprise edition and licensing (#1390)
* Add enterprise edition and licensing

* Cypress test

* License wording change

* Fix typos
2020-08-14 11:23:55 +02:00