* Allow a placeholder to be used in place of a select statement in a union all
* Updated the hogql parser version
* Use new hogql-parser version
* Fixed tests
* Update query snapshots
* Update query snapshots
* Rejigged the g4 file
* Updated mypy
* Updated mypy
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Add Celery queues env file with default queues
Reasoning:
We need to configure Celery workers in several places to consume
from a specific set of queues.
* Define some queues
* Upgrade pydantic and all related
* Upgrade mypy
* Add mypy-baseline
To update baseline when you fix something (only then!) use:
[mypy cmd] | mypy-baseline sync
* just code for migrations
* use all timezones because update would cause common_timezones to be less inclusive
* install new dependencies
* add comment
* revert
* restore
* fix type
* refactor(batch-exports): upgrade temporalio dep and use their built in metrics support
* fix: Remove unused imports
---------
Co-authored-by: Tomás Farías Santana <tomas@tomasfarias.dev>
* fix: Checkout master before checking for hogql changes
* fix: But go back to branch after done
* fix: Instead just diff with origin/master
* fix: Install libantlr, what's the worst that can happen?
* fix: Install antlr the hogql-way
* fix: Let's just checkout everything
* feat(batch-exports): Add backfill model and service support
* test(batch-exports-backfills): Add Workflow test
* chore(batch-exports-backfill): Bump migration
* feat(batch-exports): Add RedshiftBatchExportWorkflow
* feat(batch-exports): Add Redshift to BatchExport destinations
* feat(batch-exports): Support properties_data_type Redshift plugin parameter
* refactor(batch-exports): Insert rows instead of using COPY
* fix: Remove unused migration
* chore: Require aiokafka
* feat: Implement new structlog batch exports logger
* refactor: Use new structlog logger everywhere
* test: Add tests, fix things
* fix: Remove old tests
* chore: Change typing of return logger
* chore: Bump structlog
* chore: Extend docstrings
* fix: Don't use async logging as it's unsupported by temporal runtime
* test: Add logger tests
* fix: Mix pytestmark lists
* fix: Remove unused imports
* fix: Cleanup pytest warnings
* fix: Create and drop dataset for bigquery tests
* fix: Typing issue?
* fix: Let's just checkout everything
* fix: Use global event loop in tests
* fix: Blow-up cache
* fix: Truncate only if table exists
* revert-me: Skip all postgres tests
* fix: Connect to kafka in localhost
* fix: Lazily connect to Kafka
* fix: Resolve conflicts
* fix: Capture temporal context once and bind it to the logger
* fix: Make configure logger sync
* fix: Keep strong reference to background tasks
* fix: Continue consuming from log queue even if we fail to produce
* fix: Also catch the producer not starting
* fix: Remove unused await
* fix: Log kafka producer error after logger is configured
* chore(hogql): Use the C++ parser in 50% of queries
* Ensure that C++ parser is used in tests deterministically
* Align table args behavior with Python
* Bump parser version to 1.0.1
* Use new hogql-parser version
* Don't add notices for internal expressions, like in Python
* Bump parser version to 1.0.2
* Use new hogql-parser version
* Update posthog/hogql/parser.py
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* chore(hogql): Be defensive against NULLs in the C++ parser
* Clean up on C++ exceptions
* Add to CONTRIBUTING guide
* Revert `AllowShortFunctionsOnASingleLine` change
* Update HogQLX additions too
* Bump version to 1.0.0
* Use new hogql-parser version
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Allow placeholders in place of ratio expressions in hogql
* Added C++ changes for RatioExpr
* Bump `hogql_parser` version
* Upgrade ANTLR version to align with brew latest
* Update query snapshots
* Fix parser change message body
The quotes weren't right, so the step failed instead of posting a comment.
* Revert "Update query snapshots"
This reverts commit 1302d3b24f.
* Update MD5 sum
* Bump version again
* Use new hogql-parser version
---------
Co-authored-by: Michael Matloka <michal@matloka.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Michael Matloka <dev@twixes.com>
* Add partial C++ parser of HogQL
* Support all the rules and add proper error handling
* Use `AlignAfterOpenBracket: BlockIndent`
* Reuse all the parser tests for the C++ backend
* Fix relationship between headers and implementations
* Add more error handling and fix minor issues
* Build both Python and C++ parsers in the package.json script
* Update ARRAY JOIN error assertion
* Improve timeit
* Move the C extension to the top level
* Refactor `vector_to_list_string`
* Build the parser on Linux
* Build wheels for the parser
* Simplify Linux build and fix macOS
* Fix Homebrew paths on x86 and don't fail fast in CI
* Set MACOSX_DEPLOYMENT_TARGET for C++20
* Set up QEMU for Linux ARM builds
* Publish the wheels on PyPI
* Avoiding Linux ARM emulation in CI for now
* Build sdist too
* Revert Dockerfile changes
* Fix PyPI publish
* Add README and optimize sdist build
* Use setup.py directly instead of build
* Use PyPI hogql-parser instead of local
* Also revert production-unit.Dockerfile
* Fix sdist upload and add Linux ARM back
* No Linux ARM build in the end
* Fix artifact uploading
* Do try building Linux ARM
We need this for prod.
* Use `npm` in `grammar:build`
`pnpm` is not available in that job.
* Fix formatting of hogql_parser
* Build everything on macOS
* Revert "Build everything on macOS"
Not so fast actually.
* Use hogql-parser=0.1.1
* Fix dylib in macOS wheel
* Bump hogql-parser version
* Fix missing module error
* Delete timeit.py
* Make error handling robust
* Format the C++
* Use `hogql-parser==0.1.1`
* Fix reserved keyword error assertions
* Use HEAD hogql_paresr in CI
* Fix `apt` usage
* Add some sudo in CI
* Ensure package will be releasable before build
* Bump version to 0.1.3
* Cover C++ `unquote_string` with tests
* Use BuildJet ARM runners for ARM builds
* Add some instructions
* Add HogQL version check to backend CI
* Update requirements.txt
* Use `setuptools` instead of the deprecated `distutils`
* Fix working dir in backend CI
* Align ANTLR versions
* Add test for "mismatched input"
This is thrown differently than other HogQLSyntaxExceptions in C++, so might help reveal what's going on with tests failing only on Linux CI and not macOS dev
* Add types and bump version
* Comment instead of failing version check
* Automate hogql-release version bump
* Fix checkout token
* Don't build hogql-parser if there were no changes
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Update query snapshots
* Improve documentation
* Use new hogql-parser version
* Fix error start and end initialization
* Note `antlr4-cpp-runtime`
Co-authored-by: Marius Andra <marius.andra@gmail.com>
* Also remove NUL chars in C++
* Check ANTLR4 runtime archive checksum for security
* Note more decrefs to add
* Add vector size checks
* Use new hogql-parser version
* Don't support the `start` arg in C++ `parse_expr`
* Use new hogql-parser version
---------
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Marius Andra <marius.andra@gmail.com>
Co-authored-by: Tiina Turban <tiina303@gmail.com>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Ben White <ben@posthog.com>
* chore(batch-exports): add snowflake export workflow
This workflow uses Snowflake internal stages to load data from
ClickHouse into a Snowflake table. We maintain the existing events table
schema as used in the existing Snowflake App.
Something I haven't done yet is:
1. made sure e.g. we get the `elements` and `person_set` etc. data into
Snowflake.
2. the additional frontend to enable configuring the Snowflake
connection.
* remove unsed var
* include excluded events in test
