0
0
mirror of https://github.com/PostHog/posthog.git synced 2024-12-01 04:12:23 +01:00
posthog/ee/settings.py

101 lines
4.4 KiB
Python
Raw Normal View History

Personal API keys and Zapier integration (#1281) * Add missing migration * Add generate_random_token() model util * Move PublicTokenAuthentication to utils * Make use of generate_random_token * Add User.personal_access_token field * Add PersonalAccessTokenAuthentication * Fix PublicTokenAuthentication * Fix migration and auth import * Add personal_access_token to user API * Update Setup.js * Support trailing slash in API * Improve PAT auth quality * Add django-rest-hooks requirement * Update settings.py for rest_hooks * Fix django-rest-hooks requirement * Bring back API routes with no double trailing slash * Rename posthog.api.team to team_user * Add API TODO * Ad PAT auth with X-PAT HTTP header * Replace User.personal_access_token with PersonalAPIKey model * Fix PersonalAPIKey max_lengths * Describe posthog.models.utils.generate_random_token better * Add personal_api_key to API * Add authenticate_header to PersonalAPIKeyAuthentication * Add hook API endpoint * Use django.utils.timezone in place of datetime.datetime * Add Personal API Keys to Setup * Sort personal_api_keys in ORM * Add Action.on_perform() * Remove requirements.txt comment * Add a * Add REST hook tasks * Optimize PersonalAPIKeyAuthentication query * Add a trailing slash version of /e endpoint * Add team field to PersonalAPIKey model * Add personal API key support to capture endpoint, get_cached_from_token * Reject personal API keys from inactive users * Add extra_properties_json field to /capture * Improve PAK auth header regex * Use custom hook model * Deliver hooks * Handle action.on_perform * Consolidate userLogic in userLogic.tsx * Update PersonalAPIKeys.js * Make PersonalAPIKey foreign keys read-only * Update requirements/dev.txt * Make PersonalAPIKeys TSX * Fix conflict * Fix migration * Fix minor mishaps * Update and fix tests * Use CharField of random 32 bits as hook.id * Fix conflicting migrations * Fix ValidationError in HookSerializer.validate_event * Use query param in /api/event/actions ID filtering * Rename endpoint `hook` to `hooks` * Satisfy mypy * Add tests * Use DRF serialization in action_defined and annotation_created triggers * Update migration leafs * Make mypy ignore rest_hooks * Update Django signal receiver names * Update TS dependencies * Revert "Update TS dependencies" This reverts commit 7fc26fefcdc16e630e1c8fd2c510fd323d97169f. * Add field user to Hook model * Update migration leafs * Fix circular import * Fix some code * Install git before running pip install in Dockerfiles * Improve personal API keys UI * Satisfy mypy * Reword key label placeholder * Add personal API key support to /api/user/* Unfortunately these endpoints are still limited by CSRF protections at the moment, so not accessible outside PostHog itself. * Improve PersonalAPIKeyAuthentication and add CsrfOrKeyViewMiddleware * Run collectstatic before test * Don't install dev dependencies in CI * Update dependency installation order in CI * Fix bug and describe PersonalAPIKeyAuthentication * Fix CI issues * Fix typing issues * Fix more typing issues * Use /api/personal_api_keys to list keys * Move REST hooks (and therefore Zapier) to ee/ * Refactor personal API logic with kea-loaders * Add "More about API authentication in PostHog docs." * Update PersonalAPIKeys.tsx * Use TestMixin * Fix "Authentication" that should've been "Authorization" * Add option to skip self.client.force_login in API tests * Include team_id and user_id in personal API key serialization * Update test_hooks.py * Add personal API key tests * Remove leftover * Make ee.settings override posthog.settings * Don't directly import from models * Remove unused imports * Fix mypy issues * Fix HOOK_DELIVERER * Use decorator for /api/user PAK auth * Don't fire REST hook if user doesn't have "zapier" feature * Import Optional * Reword to "premium Zapier" * Make mypy happy * Fix test_delete_personal_api_key * Fix misclick * Fix and test /capture with personal API key * Make mypy happy * Remove extra_properties_json * Resolve migrations * Remove apt-utils * Optimize and test PAK user.is_active filtering * Replace DEBUG true with 1 * Remove unused instance_id * Improve typing * Fix deletion toast * Refactor CopyToClipboard and use it in PAKs * Use toast.success * Update migrations * Fix migration * Fix migrations * Complete merge Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-08-26 10:34:57 +02:00
"""
Django settings for PostHog Enterprise Edition.
"""
import os
from typing import Dict, List
from ee.kafka_client.topics import KAFKA_EVENTS_PLUGIN_INGESTION as DEFAULT_KAFKA_EVENTS_PLUGIN_INGESTION
from posthog.constants import AnalyticsDBMS
2021-08-27 16:20:49 +02:00
from posthog.settings import AUTHENTICATION_BACKENDS, PRIMARY_DB, SITE_URL, TEST, get_from_env
2021-08-27 18:37:30 +02:00
from posthog.utils import str_to_bool
Personal API keys and Zapier integration (#1281) * Add missing migration * Add generate_random_token() model util * Move PublicTokenAuthentication to utils * Make use of generate_random_token * Add User.personal_access_token field * Add PersonalAccessTokenAuthentication * Fix PublicTokenAuthentication * Fix migration and auth import * Add personal_access_token to user API * Update Setup.js * Support trailing slash in API * Improve PAT auth quality * Add django-rest-hooks requirement * Update settings.py for rest_hooks * Fix django-rest-hooks requirement * Bring back API routes with no double trailing slash * Rename posthog.api.team to team_user * Add API TODO * Ad PAT auth with X-PAT HTTP header * Replace User.personal_access_token with PersonalAPIKey model * Fix PersonalAPIKey max_lengths * Describe posthog.models.utils.generate_random_token better * Add personal_api_key to API * Add authenticate_header to PersonalAPIKeyAuthentication * Add hook API endpoint * Use django.utils.timezone in place of datetime.datetime * Add Personal API Keys to Setup * Sort personal_api_keys in ORM * Add Action.on_perform() * Remove requirements.txt comment * Add a * Add REST hook tasks * Optimize PersonalAPIKeyAuthentication query * Add a trailing slash version of /e endpoint * Add team field to PersonalAPIKey model * Add personal API key support to capture endpoint, get_cached_from_token * Reject personal API keys from inactive users * Add extra_properties_json field to /capture * Improve PAK auth header regex * Use custom hook model * Deliver hooks * Handle action.on_perform * Consolidate userLogic in userLogic.tsx * Update PersonalAPIKeys.js * Make PersonalAPIKey foreign keys read-only * Update requirements/dev.txt * Make PersonalAPIKeys TSX * Fix conflict * Fix migration * Fix minor mishaps * Update and fix tests * Use CharField of random 32 bits as hook.id * Fix conflicting migrations * Fix ValidationError in HookSerializer.validate_event * Use query param in /api/event/actions ID filtering * Rename endpoint `hook` to `hooks` * Satisfy mypy * Add tests * Use DRF serialization in action_defined and annotation_created triggers * Update migration leafs * Make mypy ignore rest_hooks * Update Django signal receiver names * Update TS dependencies * Revert "Update TS dependencies" This reverts commit 7fc26fefcdc16e630e1c8fd2c510fd323d97169f. * Add field user to Hook model * Update migration leafs * Fix circular import * Fix some code * Install git before running pip install in Dockerfiles * Improve personal API keys UI * Satisfy mypy * Reword key label placeholder * Add personal API key support to /api/user/* Unfortunately these endpoints are still limited by CSRF protections at the moment, so not accessible outside PostHog itself. * Improve PersonalAPIKeyAuthentication and add CsrfOrKeyViewMiddleware * Run collectstatic before test * Don't install dev dependencies in CI * Update dependency installation order in CI * Fix bug and describe PersonalAPIKeyAuthentication * Fix CI issues * Fix typing issues * Fix more typing issues * Use /api/personal_api_keys to list keys * Move REST hooks (and therefore Zapier) to ee/ * Refactor personal API logic with kea-loaders * Add "More about API authentication in PostHog docs." * Update PersonalAPIKeys.tsx * Use TestMixin * Fix "Authentication" that should've been "Authorization" * Add option to skip self.client.force_login in API tests * Include team_id and user_id in personal API key serialization * Update test_hooks.py * Add personal API key tests * Remove leftover * Make ee.settings override posthog.settings * Don't directly import from models * Remove unused imports * Fix mypy issues * Fix HOOK_DELIVERER * Use decorator for /api/user PAK auth * Don't fire REST hook if user doesn't have "zapier" feature * Import Optional * Reword to "premium Zapier" * Make mypy happy * Fix test_delete_personal_api_key * Fix misclick * Fix and test /capture with personal API key * Make mypy happy * Remove extra_properties_json * Resolve migrations * Remove apt-utils * Optimize and test PAK user.is_active filtering * Replace DEBUG true with 1 * Remove unused instance_id * Improve typing * Fix deletion toast * Refactor CopyToClipboard and use it in PAKs * Use toast.success * Update migrations * Fix migration * Fix migrations * Complete merge Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-08-26 10:34:57 +02:00
# Zapier REST hooks
HOOK_EVENTS: Dict[str, str] = {
Personal API keys and Zapier integration (#1281) * Add missing migration * Add generate_random_token() model util * Move PublicTokenAuthentication to utils * Make use of generate_random_token * Add User.personal_access_token field * Add PersonalAccessTokenAuthentication * Fix PublicTokenAuthentication * Fix migration and auth import * Add personal_access_token to user API * Update Setup.js * Support trailing slash in API * Improve PAT auth quality * Add django-rest-hooks requirement * Update settings.py for rest_hooks * Fix django-rest-hooks requirement * Bring back API routes with no double trailing slash * Rename posthog.api.team to team_user * Add API TODO * Ad PAT auth with X-PAT HTTP header * Replace User.personal_access_token with PersonalAPIKey model * Fix PersonalAPIKey max_lengths * Describe posthog.models.utils.generate_random_token better * Add personal_api_key to API * Add authenticate_header to PersonalAPIKeyAuthentication * Add hook API endpoint * Use django.utils.timezone in place of datetime.datetime * Add Personal API Keys to Setup * Sort personal_api_keys in ORM * Add Action.on_perform() * Remove requirements.txt comment * Add a * Add REST hook tasks * Optimize PersonalAPIKeyAuthentication query * Add a trailing slash version of /e endpoint * Add team field to PersonalAPIKey model * Add personal API key support to capture endpoint, get_cached_from_token * Reject personal API keys from inactive users * Add extra_properties_json field to /capture * Improve PAK auth header regex * Use custom hook model * Deliver hooks * Handle action.on_perform * Consolidate userLogic in userLogic.tsx * Update PersonalAPIKeys.js * Make PersonalAPIKey foreign keys read-only * Update requirements/dev.txt * Make PersonalAPIKeys TSX * Fix conflict * Fix migration * Fix minor mishaps * Update and fix tests * Use CharField of random 32 bits as hook.id * Fix conflicting migrations * Fix ValidationError in HookSerializer.validate_event * Use query param in /api/event/actions ID filtering * Rename endpoint `hook` to `hooks` * Satisfy mypy * Add tests * Use DRF serialization in action_defined and annotation_created triggers * Update migration leafs * Make mypy ignore rest_hooks * Update Django signal receiver names * Update TS dependencies * Revert "Update TS dependencies" This reverts commit 7fc26fefcdc16e630e1c8fd2c510fd323d97169f. * Add field user to Hook model * Update migration leafs * Fix circular import * Fix some code * Install git before running pip install in Dockerfiles * Improve personal API keys UI * Satisfy mypy * Reword key label placeholder * Add personal API key support to /api/user/* Unfortunately these endpoints are still limited by CSRF protections at the moment, so not accessible outside PostHog itself. * Improve PersonalAPIKeyAuthentication and add CsrfOrKeyViewMiddleware * Run collectstatic before test * Don't install dev dependencies in CI * Update dependency installation order in CI * Fix bug and describe PersonalAPIKeyAuthentication * Fix CI issues * Fix typing issues * Fix more typing issues * Use /api/personal_api_keys to list keys * Move REST hooks (and therefore Zapier) to ee/ * Refactor personal API logic with kea-loaders * Add "More about API authentication in PostHog docs." * Update PersonalAPIKeys.tsx * Use TestMixin * Fix "Authentication" that should've been "Authorization" * Add option to skip self.client.force_login in API tests * Include team_id and user_id in personal API key serialization * Update test_hooks.py * Add personal API key tests * Remove leftover * Make ee.settings override posthog.settings * Don't directly import from models * Remove unused imports * Fix mypy issues * Fix HOOK_DELIVERER * Use decorator for /api/user PAK auth * Don't fire REST hook if user doesn't have "zapier" feature * Import Optional * Reword to "premium Zapier" * Make mypy happy * Fix test_delete_personal_api_key * Fix misclick * Fix and test /capture with personal API key * Make mypy happy * Remove extra_properties_json * Resolve migrations * Remove apt-utils * Optimize and test PAK user.is_active filtering * Replace DEBUG true with 1 * Remove unused instance_id * Improve typing * Fix deletion toast * Refactor CopyToClipboard and use it in PAKs * Use toast.success * Update migrations * Fix migration * Fix migrations * Complete merge Co-authored-by: Tim Glaser <tim@glsr.nl>
2020-08-26 10:34:57 +02:00
# "event_name": "App.Model.Action" (created/updated/deleted)
"action_defined": "posthog.Action.created_custom",
"action_performed": "posthog.Action.performed",
"annotation_created": "posthog.Annotation.created_custom",
}
HOOK_FINDER = "ee.models.hook.find_and_fire_hook"
HOOK_DELIVERER = "ee.models.hook.deliver_hook_wrapper"
# Social auth
SOCIAL_AUTH_GOOGLE_OAUTH2_KEY = os.getenv("SOCIAL_AUTH_GOOGLE_OAUTH2_KEY")
SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET = os.getenv("SOCIAL_AUTH_GOOGLE_OAUTH2_SECRET")
if "SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS" in os.environ:
SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS: List[str] = os.environ[
"SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS"
].split(",")
2021-08-20 19:01:33 +02:00
AUTHENTICATION_BACKENDS = AUTHENTICATION_BACKENDS + [
"social_core.backends.google.GoogleOAuth2",
]
2021-08-27 16:20:49 +02:00
# SAML
2021-09-14 11:43:26 +02:00
SAML_DISABLED = get_from_env("SAML_DISABLED", False, type_cast=str_to_bool)
2021-08-27 16:20:49 +02:00
SAML_CONFIGURED = False
SOCIAL_AUTH_SAML_SP_ENTITY_ID = SITE_URL
SOCIAL_AUTH_SAML_SECURITY_CONFIG = {
"wantAttributeStatement": False, # AttributeStatement is optional in the specification
}
# Attributes below are required for the SAML integration from social_core to work properly
SOCIAL_AUTH_SAML_SP_PUBLIC_CERT = ""
SOCIAL_AUTH_SAML_SP_PRIVATE_KEY = ""
SOCIAL_AUTH_SAML_ORG_INFO = {"en-US": {"name": "posthog", "displayname": "PostHog", "url": "https://posthog.com"}}
SOCIAL_AUTH_SAML_TECHNICAL_CONTACT = {"givenName": "PostHog Support", "emailAddress": "hey@posthog.com"}
SOCIAL_AUTH_SAML_SUPPORT_CONTACT = SOCIAL_AUTH_SAML_TECHNICAL_CONTACT
def getenv(key, default=""):
"""
This is prevent a bug in helm deploys where the env var is not returning the default
because os.getenv is returning an empty string ''
"""
val = os.getenv(key)
if val == None or val == "":
return default
return val
2021-08-27 16:20:49 +02:00
# Set settings only if SAML is enabled
2021-09-14 11:43:26 +02:00
if not SAML_DISABLED and os.getenv("SAML_ENTITY_ID") and os.getenv("SAML_ACS_URL") and os.getenv("SAML_X509_CERT"):
2021-08-27 16:20:49 +02:00
SAML_CONFIGURED = True
AUTHENTICATION_BACKENDS = AUTHENTICATION_BACKENDS + [
"social_core.backends.saml.SAMLAuth",
]
SOCIAL_AUTH_SAML_ENABLED_IDPS = {
"posthog_custom": {
"entity_id": getenv("SAML_ENTITY_ID"),
"url": getenv("SAML_ACS_URL"),
"x509cert": getenv("SAML_X509_CERT"),
"attr_user_permanent_id": getenv("SAML_ATTR_PERMANENT_ID", "name_id"),
"attr_first_name": getenv("SAML_ATTR_FIRST_NAME", "first_name"),
"attr_last_name": getenv("SAML_ATTR_LAST_NAME", "last_name"),
"attr_email": getenv("SAML_ATTR_EMAIL", "email"),
2021-08-27 16:20:49 +02:00
},
}
2021-08-27 18:37:30 +02:00
SAML_ENFORCED = get_from_env("SAML_ENFORCED", False, type_cast=str_to_bool)
2021-08-27 16:20:49 +02:00
# ClickHouse and Kafka
KAFKA_ENABLED = PRIMARY_DB == AnalyticsDBMS.CLICKHOUSE and not TEST
# Schedule to run column materialization on. Follows crontab syntax.
# Use empty string to prevent from materializing
MATERIALIZE_COLUMNS_SCHEDULE_CRON = get_from_env("MATERIALIZE_COLUMNS_SCHEDULE_CRON", "0 5 * * SAT")
# Minimum query time before a query if considered for optimization by adding materialized columns
MATERIALIZE_COLUMNS_MINIMUM_QUERY_TIME = get_from_env("MATERIALIZE_COLUMNS_MINIMUM_QUERY_TIME", 3000, type_cast=int)
# How many hours backwards to look for queries to optimize
MATERIALIZE_COLUMNS_ANALYSIS_PERIOD_HOURS = get_from_env(
"MATERIALIZE_COLUMNS_ANALYSIS_PERIOD_HOURS", 7 * 24, type_cast=int
)
# How big of a timeframe to backfill when materializing event properties. 0 for no backfilling
MATERIALIZE_COLUMNS_BACKFILL_PERIOD_DAYS = get_from_env("MATERIALIZE_COLUMNS_BACKFILL_PERIOD_DAYS", 90, type_cast=int)
# Maximum number of columns to materialize at once. Avoids running into resource bottlenecks (storage + ingest + backfilling).
MATERIALIZE_COLUMNS_MAX_AT_ONCE = get_from_env("MATERIALIZE_COLUMNS_MAX_AT_ONCE", 10, type_cast=int)
# Topic to write events to between clickhouse
KAFKA_EVENTS_PLUGIN_INGESTION_TOPIC: str = os.getenv(
"KAFKA_EVENTS_PLUGIN_INGESTION_TOPIC", DEFAULT_KAFKA_EVENTS_PLUGIN_INGESTION
)