0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-11-30 19:41:46 +01:00
Commit Graph

10 Commits

Author SHA1 Message Date
David Newell
4e22252235
chore: run clustering in background task (#21080) 2024-03-28 12:12:51 +00:00
David Newell
4f6d9c8673
feat: generate recording text embeddings (#20046)
* make migration

* general flow

* abstract shared methods

* generate input

* remove postgres migration

* generate embedding strings

* remove random file

* Update query snapshots

* Update query snapshots

* feat: create periodic replay embedding

* first sketch of table

* batch and flush embeddings

* add default to timestamp generation

* fetch recordings query

* save first embeddings to CH

* dump session metadata into tokens

* fix lint

* brain dump to help th future traveller

* prom timing instead

* fix input concatenation

* add an e :/

* obey mypy

* some time limits to reduce what we query

* a little fiddling to get it to run locally

* paging and counting

* Update query snapshots

* Update query snapshots

* move the AI stuff to EE for now

* Update query snapshots

* kick off the task with interval from settings

* push embedding generation onto its own queue

* on a different queue

* EE to the max

* doh

* fix

* fangling

* Remove clashes so we can merge this into the other PR

* Remove clashes so we can merge this into the other PR

* start wiring up Celery task

* hmmm

* it's a chord

* wire up celery simple version

* rename

* why is worker failing

* Update .run/Celery.run.xml

* update embedding input to remove duplicates

* ttl on the table

* Revert "update embedding input to remove duplicates"

This reverts commit 9a09d9c9f0.

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Paul D'Ambra <paul@posthog.com>
2024-02-14 12:50:42 +00:00
Tom Owers
5513be7731
chore: upgraded Ruff linter (#18188)
* Upgraded Ruff linter

* Formatted whole codebase with new ruff rules

* Revert import removal

* Fixed mypi issues or added ignores

* Fixed schema formatting

* Fixed hogvm failing tests

* Remove duplicate key in list
2023-10-26 12:38:15 +02:00
Paul D'Ambra
a87b247cd3
feat: step towards feature folders for replay backend (#17484)
* feat: step towards feature folders for replay backend

* Update query snapshots

* Update query snapshots

* that got mypy very excited

* fix

* fix

* fix

* dnagling snapshot

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2023-09-18 13:35:59 +01:00
Ben White
dc3e1ed9e3
feat: Persist pinned recordings to S3 (#13473) 2023-01-12 09:23:51 +01:00
Eric Duong
bf81161e7f
refactor: Fix periodic tasks (#10491)
* Revert "fix: Fix ee celery tasks (#10481)"

This reverts commit 5b3fbbd57b.

* chore: empty

* fix: move ee tasks to celery.py

* chore: empty

* remove file

* remove init
2022-06-24 13:52:32 -04:00
Eric Duong
8c6923a0e0
fix: Revert "fix: Fix ee celery tasks" (#10487)
* Revert "fix: Fix ee celery tasks (#10481)"

This reverts commit 5b3fbbd57b.

* chore: empty
2022-06-24 15:58:37 +00:00
Ben White
5b3fbbd57b
fix: Fix ee celery tasks (#10481)
* fix: Fix ee celery tasks
2022-06-24 17:03:26 +02:00
Eric Duong
cb485164f7
refactor(foss): materialize and celery (#10434)
* refactor: foss move properties to /posthog

* refactor: move replication and materializtaion

* refactor: move file

* refactor: move test and journeys

* refactor: move breakdown props

* refactor: move query imports

* refactor: move more ee dependencies

* refactor: restore groupsjoinquery

* fix: errors

* refactor: no ee.clickhouse dependncies

* try import tasks

* refactor: move materialization

* refactor: change foss split for column optimizer

* run black

* fix: imports

* remove comment

* Update snapshots

* run black

* skip isort

* Update snapshots

* format

* more fixes

* refactor(FOSS): split out paths query code (#10378)

* refactor: move migrations

* refactor: move idl

* fix: move more imports

* import adjustments

* fix: test import

* fix: test import

* fix: remove restriction

* refactor: split out paths query code

* refactor: more code splitting

* fix: types

* refactor(FOSS): Setup such that FOSS is deployable (#10352)

* refactor: move migrations

* refactor: move idl

* fix: move more imports

* import adjustments

* fix: test import

* fix: test import

* fix: remove restriction

* fix import

* refactor: add snapshost

* subscription-import

* refactor: new file

* fix: remove unused

* use app object import

* Update snapshots

* ensure same app is used

* use decorator

* remove file

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-06-23 20:17:39 -04:00
Michael Matloka
c040601f49
Personal API keys and Zapier integration (#1281)
* Add missing migration

* Add generate_random_token() model util

* Move PublicTokenAuthentication to utils

* Make use of generate_random_token

* Add User.personal_access_token field

* Add PersonalAccessTokenAuthentication

* Fix PublicTokenAuthentication

* Fix migration and auth import

* Add personal_access_token to user API

* Update Setup.js

* Support trailing slash in API

* Improve PAT auth quality

* Add django-rest-hooks requirement

* Update settings.py for rest_hooks

* Fix django-rest-hooks requirement

* Bring back API routes with no double trailing slash

* Rename posthog.api.team to team_user

* Add API TODO

* Ad PAT auth with X-PAT HTTP header

* Replace User.personal_access_token with PersonalAPIKey model

* Fix PersonalAPIKey max_lengths

* Describe posthog.models.utils.generate_random_token better

* Add personal_api_key to API

* Add authenticate_header to PersonalAPIKeyAuthentication

* Add hook API endpoint

* Use django.utils.timezone in place of datetime.datetime

* Add Personal API Keys to Setup

* Sort personal_api_keys in ORM

* Add Action.on_perform()

* Remove requirements.txt comment

* Add a

* Add REST hook tasks

* Optimize PersonalAPIKeyAuthentication query

* Add a trailing slash version of /e endpoint

* Add team field to PersonalAPIKey model

* Add personal API key support to capture endpoint, get_cached_from_token

* Reject personal API keys from inactive users

* Add extra_properties_json field to /capture

* Improve PAK auth header regex

* Use custom hook model

* Deliver hooks

* Handle action.on_perform

* Consolidate userLogic in userLogic.tsx

* Update PersonalAPIKeys.js

* Make PersonalAPIKey foreign keys read-only

* Update requirements/dev.txt

* Make PersonalAPIKeys TSX

* Fix conflict

* Fix migration

* Fix minor mishaps

* Update and fix tests

* Use CharField of random 32 bits as hook.id

* Fix conflicting migrations

* Fix ValidationError in HookSerializer.validate_event

* Use query param in /api/event/actions ID filtering

* Rename endpoint `hook` to `hooks`

* Satisfy mypy

* Add tests

* Use DRF serialization in action_defined and annotation_created triggers

* Update migration leafs

* Make mypy ignore rest_hooks

* Update Django signal receiver names

* Update TS dependencies

* Revert "Update TS dependencies"

This reverts commit 7fc26fefcd.

* Add field user to Hook model

* Update migration leafs

* Fix circular import

* Fix some code

* Install git before running pip install in Dockerfiles

* Improve personal API keys UI

* Satisfy mypy

* Reword key label placeholder

* Add personal API key support to /api/user/*

Unfortunately these endpoints are still limited by CSRF protections at the moment, so not accessible outside PostHog itself.

* Improve PersonalAPIKeyAuthentication and add CsrfOrKeyViewMiddleware

* Run collectstatic before test

* Don't install dev dependencies in CI

* Update dependency installation order in CI

* Fix bug and describe PersonalAPIKeyAuthentication

* Fix CI issues

* Fix typing issues

* Fix more typing issues

* Use /api/personal_api_keys to list keys

* Move REST hooks (and therefore Zapier) to ee/

* Refactor personal API logic with kea-loaders

* Add "More about API authentication in PostHog docs."

* Update PersonalAPIKeys.tsx

* Use TestMixin

* Fix "Authentication" that should've been "Authorization"

* Add option to skip self.client.force_login in API tests

* Include team_id and user_id in personal API key serialization

* Update test_hooks.py

* Add personal API key tests

* Remove leftover

* Make ee.settings override posthog.settings

* Don't directly import from models

* Remove unused imports

* Fix mypy issues

* Fix HOOK_DELIVERER

* Use decorator for /api/user PAK auth

* Don't fire REST hook if user doesn't have "zapier" feature

* Import Optional

* Reword to "premium Zapier"

* Make mypy happy

* Fix test_delete_personal_api_key

* Fix misclick

* Fix and test /capture with personal API key

* Make mypy happy

* Remove extra_properties_json

* Resolve migrations

* Remove apt-utils

* Optimize and test PAK user.is_active filtering

* Replace DEBUG true with 1

* Remove unused instance_id

* Improve typing

* Fix deletion toast

* Refactor CopyToClipboard and use it in PAKs

* Use toast.success

* Update migrations

* Fix migration

* Fix migrations

* Complete merge

Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-08-26 10:34:57 +02:00