Skip to content

fix(signals): GitHub Issues source emits nothing when the repo name has a hyphen #95346

Description

@andrewm4894

Problem

A team turns on the GitHub Issues inbox source, the card reads Synced, the issues table syncs every few hours, and no report ever comes out of it. Nothing in the product says anything went wrong.

The warehouse side is healthy. What dies is the signal-emission child workflow that runs after each sync, and it dies before it can log anything. It has done so on every sync, for every team whose GitHub repository name contains a hyphen, since repo-qualified GitHub schema names landed in #71553.

A community member reported it in Discord ("I have it already setup but it never created a PR from a GH issue yet"). Internal thread: https://posthog.slack.com/archives/C09SK2PAGKF/p1788542132947169

What happens today

After a warehouse sync completes, external-data-job starts emit-data-import-signals-<job_id> on the signals worker (video-export-task-queue). The activity calls data_warehouse_record_fetcher in products/signals/backend/emission/fetchers/data_warehouse.py, which builds the HogQL query by f-string:

query = f"""
    SELECT {fields_sql}
    FROM {table_name}
    WHERE {where_sql}
    LIMIT {config.max_records}
"""

table_name comes from get_data_warehouse_table_name(schema.source, schema.table.name). For a GitHub source that is github.<owner>_<repo>__issues, so a repository such as andrewm4894/andys-daily-factoids produces:

FROM github.andrewm4894_andys-daily-factoids__issues

HogQL tokenises the hyphens as subtraction and parse_select raises:

SyntaxError: trailing tokens after expression: '-' (Dash)

parse_select runs before the try/except that logs Error querying new records, so:

  • nothing is logged and no PostHog event fires
  • Temporal retries the activity three times and the child workflow fails
  • the parent sync has already succeeded and advanced last_synced_at, so the issues in that window are never revisited

The only trace is three Starting signal emission for Github/... lines one to two seconds apart per sync on temporal-worker-video-export, each followed by Querying new records for signal emission and then silence.

The comment above the f-string says none of the data comes from external input. That is no longer true: the customer's repository name is in the table key.

Most enabled GitHub Issues sources have a hyphen in the repository name, and none of those teams has ever emitted a github signal. Teams with plain repository names emit normally.

Repository names with a dot (owner/my.site) probably fail a second way: Database.get_table_node splits the joined name on ., so the key resolves to a three-part chain and Unknown table. That one is inside the try and would be logged. Unverified.

Same shape, different source: Intercom

products/signals/backend/emission/intercom_conversations.py sets:

partition_field="fromUnixTimestamp(toUInt32(created_at))",

toUInt32 is not a HogQL function, so every emission run for the Intercom conversations source fails with:

QueryError: Unsupported function call 'toUInt32(...)'. Perhaps you meant 'toInt(...)'?

This one is logged at error level with team_id (it is inside the try), three attempts per sync, on every team with the source enabled, since #72535.

What we want

  • A synced data-import source emits signals whatever the repository name looks like.
  • A fetch that cannot even be parsed is logged and counted like a fetch that fails at execution.
  • Every registered source's partition_field and where_clause is known to print through HogQL before it ships.
  • Teams that were silently skipped can be caught up without waiting for a new issue to be opened.

Design

Quote the table identifier

In data_warehouse_record_fetcher, split table_name on . and pass each segment through escape_hogql_identifier (posthog/hogql/escape_sql.py), then join with .. The escaper leaves plain identifiers alone and backticks anything else, so existing sources print byte-identically and github.andrewm4894_andys-daily-factoids__issues becomes:

FROM github.`andrewm4894_andys-daily-factoids__issues`

Apply the same to google_search_console_record_fetcher in google_search_console_opportunities.py, which has the same f-string.

Check whether a two-part chain whose second segment contains a dot resolves through Database.get_table_node; if not, the dotted-repo shape needs its own handling and the test below should cover it.

Move parse_select inside the try

So a parse failure reaches logger.exception("Error querying new records: ...") with the run's labels, and the same re-raise semantics apply.

Fix the Intercom partition field

Replace toUInt32 with toInt (or whichever HogQL cast prints for the column's type), and confirm the printed expression against a real Intercom conversations table.

Guard the registry

Add a test that iterates every entry in _SIGNAL_TABLE_CONFIGS, builds the continuous-sync query the fetcher would build (with a placeholder table name), and runs it through parse_select plus print_ast against a HogQL database. Any source with a non-HogQL function in partition_field or where_clause fails the suite instead of failing in prod.

Correct the comment

Replace the "none of the data comes externally" comment with why the identifier is escaped.

Tests

  • products/signals/backend/emission/tests/test_emit_signals.py: parameterize the existing continuous-sync fetcher test over table names test_table, github.owner_my-repo__issues, and github.owner_my.repo__issues; assert the query parses and the FROM clause is quoted.
  • A parse failure (feed a table name that still cannot be quoted, or patch parse_select to raise) is logged through the same Error querying new records path and re-raised.
  • Registry-wide print test as above.
  • test_github_issues.py / a new Intercom test: the config's partition_field prints through HogQL.

Recovery

python manage.py emit_signals_from_warehouse --team-id <id> --source github --last-synced-at <iso> runs the child workflow for one team without a sync. After the fix ships, run it for affected teams with a --last-synced-at far enough back to cover the time the source was on. first_sync_lookback_days is one day, so nothing catches up by itself.

Acceptance

  • A GitHub Issues source on a repository named with a hyphen emits a signal for a newly opened issue on the next sync.
  • A parse failure in the fetcher shows up as an error log line carrying team_id, schema_id and source_type.
  • The Intercom conversations source emits on its next sync.
  • The registry print test is green for every registered source.

Out of scope

  • Surfacing emission failures on the source card (it reads Synced while every emission dies). Worth its own issue.
  • A metric or event on emission failure. Same.

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions