penmillion

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: 6dc7b32ef3eed8378f40270d35c3a0b7a45dd21b https://github.com/dreamwidth/dreamwidth/commit/6dc7b32ef3eed8378f40270d35c3a0b7a45dd21b Author: Mark Smith mark@dreamwidth.org Date: 2026-04-20 (Mon, 20 Apr 2026)

Changed paths: M cgi-bin/DW/External/Userinfo.pm

Log Message:

Drop high-cardinality username tag from extacct stats

The username:$user tag on dw.worker.extacct.{success,failure} tracked each remote external-site user individually, making it the top metric by active series count in Grafana Cloud billing. Site alone is bounded to the DW::External::Site enum and gives the actionable dimension.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

Commit: ad345841bf9b424ceb6bf65be2136b88fb612301 https://github.com/dreamwidth/dreamwidth/commit/ad345841bf9b424ceb6bf65be2136b88fb612301 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-20 (Mon, 20 Apr 2026)

Changed paths: M cgi-bin/DW/Controller/Importer.pm M cgi-bin/DW/Logic/Importer.pm

Log Message:

Validate importer hostname against source whitelist

The /tools/importer UI offered a dropdown of three allowed sources (livejournal.com, insanejournal.com, dreamwidth.org), but set_import_data_for_user accepted whatever hostname the POST carried and INSERTed it straight into import_data. A crafted POST could inject arbitrary hostnames, which then flowed into the new hostname: tag on dw.worker.importer.job_completed as a cardinality-injection vector.

Extracts the allowed-source list into DW::Logic::Importer->allowed_sources so the controller's dropdown rendering and the logic layer's validation share one definition, and rejects any hostname not in the list.

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

Compare: https://github.com/dreamwidth/dreamwidth/compare/00b8f85a98e0...ad345841bf9b

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: a19cc8c787fca5419d8469a5977323ed68c450c2 https://github.com/dreamwidth/dreamwidth/commit/a19cc8c787fca5419d8469a5977323ed68c450c2 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-12 (Sun, 12 Apr 2026)

Changed paths: M src/dwtool/main.go

Log Message:

Improve esn-trace URL resolution with userid lookup and precise Loki queries

Resolve the journal's userid from their profile page so we can build exact Loki regex queries matching [esn ETYPEID:JOURNALID:JTALKID:ARG2] instead of broad substring matches on ":JTALKID:" which produced false positives.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

Commit: b1e684b607034fa480f654db748922660b063d9d https://github.com/dreamwidth/dreamwidth/commit/b1e684b607034fa480f654db748922660b063d9d Author: Mark Smith mark@dreamwidth.org Date: 2026-04-12 (Sun, 12 Apr 2026)

Changed paths: M cgi-bin/DW/Task/ESN/FiredEvent.pm M cgi-bin/LJ/DB.pm

Log Message:

Fix ESN dropping notifications due to replication lag on comment load

FiredEvent reads comment data to determine which users' Reply subscriptions are relevant. LJ::Comment::preload_rows loads via get_cluster_reader, which hits the replica — but the comment was just written to the master and may not have replicated yet. When this race is lost, the comment fails to load, relevantuserids returns empty, Reply subscriptions are excluded, and the notification is silently dropped.

Fix: wrap the subscription lookup in LJ::DB::require_master so all reads go to the cluster master. Also extend get_cluster_reader to respect the existing PRAGMAFORCE_MASTER flag, which previously only affected global (non-cluster) readers.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

Compare: https://github.com/dreamwidth/dreamwidth/compare/eae7aaa21ba6...b1e684b60703

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: b046642d6f54f301e4f4a128472cc9edfa8fd80c https://github.com/dreamwidth/dreamwidth/commit/b046642d6f54f301e4f4a128472cc9edfa8fd80c Author: Mark Smith mark@dreamwidth.org Date: 2026-04-12 (Sun, 12 Apr 2026)

Changed paths: M .github/workflows/tasks/worker-birthday-notify-service.json M .github/workflows/tasks/worker-change-poster-id-service.json M .github/workflows/tasks/worker-codebuild-notifier-service.json M .github/workflows/tasks/worker-content-importer-lite-service.json M .github/workflows/tasks/worker-content-importer-service.json M .github/workflows/tasks/worker-content-importer-verify-service.json M .github/workflows/tasks/worker-directory-meta-service.json M .github/workflows/tasks/worker-distribute-invites-service.json M .github/workflows/tasks/worker-dw-change-poster-id-service.json M .github/workflows/tasks/worker-dw-distribute-invites-service.json M .github/workflows/tasks/worker-dw-embeds-service.json M .github/workflows/tasks/worker-dw-esn-cluster-subs-service.json M .github/workflows/tasks/worker-dw-esn-filter-subs-service.json M .github/workflows/tasks/worker-dw-esn-fired-event-service.json M .github/workflows/tasks/worker-dw-esn-process-sub-service.json M .github/workflows/tasks/worker-dw-import-eraser-service.json M .github/workflows/tasks/worker-dw-incoming-email-service.json M .github/workflows/tasks/worker-dw-latest-feed-service.json M .github/workflows/tasks/worker-dw-lazy-cleanup-service.json M .github/workflows/tasks/worker-dw-mass-privacy-service.json M .github/workflows/tasks/worker-dw-send-email-service.json M .github/workflows/tasks/worker-dw-sphinx-copier-service.json M .github/workflows/tasks/worker-dw-support-notify-service.json M .github/workflows/tasks/worker-dw-synsuck-service.json M .github/workflows/tasks/worker-dw-xpost-service.json M .github/workflows/tasks/worker-embeds-service.json M .github/workflows/tasks/worker-expunge-users-service.json M .github/workflows/tasks/worker-import-eraser-service.json M .github/workflows/tasks/worker-import-scheduler-service.json M .github/workflows/tasks/worker-incoming-email-service.json M .github/workflows/tasks/worker-latest-feed-service.json M .github/workflows/tasks/worker-lazy-cleanup-service.json M .github/workflows/tasks/worker-paidstatus-service.json M .github/workflows/tasks/worker-process-privacy-service.json M .github/workflows/tasks/worker-resolve-extacct-service.json M .github/workflows/tasks/worker-schedule-synsuck-service.json M .github/workflows/tasks/worker-ses-incoming-email-service.json M .github/workflows/tasks/worker-shop-creditcard-charge-service.json M .github/workflows/tasks/worker-spellcheck-gm-service.json M .github/workflows/tasks/worker-sphinx-copier-service.json M .github/workflows/tasks/worker-sphinx-search-gm-service.json M .github/workflows/tasks/worker-support-notify-service.json M config/update-workflows.py

Log Message:

Ship worker logs directly to Grafana Cloud Loki via Fluent Bit

Regenerated all 42 worker task definitions to use awsfirelens with a Fluent Bit log_router sidecar instead of the awslogs driver. Worker logs now go directly to Grafana Cloud Loki over HTTPS, bypassing CloudWatch Logs entirely.

Each task definition gains a log_router container (~15MB RSS) running aws-for-fluent-bit:3 (Fluent Bit v4.2.2, the 3.x line based on AL2023 — the 2.x line ships Fluent Bit 1.9 which has no Loki plugin). Loki credentials are injected via SSM secrets at task startup. The worker container's logDriver changes from awslogs to awsfirelens with the native loki output plugin.

Fluent Bit's own logs still go to CloudWatch (/dreamwidth/fluent-bit) so the log router itself can be debugged if needed.

Generated by config/update-workflows.py from config/workers.json.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: d7d5b8fbd0bb1063688f06cc820d3e65bae09a45 https://github.com/dreamwidth/dreamwidth/commit/d7d5b8fbd0bb1063688f06cc820d3e65bae09a45 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-12 (Sun, 12 Apr 2026)

Changed paths: M cgi-bin/DW/Task/ESN/ProcessSub.pm M cgi-bin/LJ/ESN.pm M etc/config-local.pl.example

Log Message:

Auto-deactivate ESN subscriptions for idle users

When processing notifications, deactivate subscriptions that belong to users who have been idle for over a year ($LJ::ESN_INACTIVE_DAYS, default 365). This happens in two places:

• LJ::ESN::unique_matching_subs: if a sub requires a paid capability the user no longer has (e.g. thread tracking after paid expired) AND the user is idle, deactivate the sub before it reaches matches_filter.

• DW::Task::ESN::ProcessSub: if a user is idle, deactivate the sub after the first skip so it never generates another SQS job.

Deactivated subs set the INACTIVE flag, NOT deleted. The user can still see them on /manage/tracking and reactivate them when they return. LJ::Subscription::create also automatically reactivates matching inactive subs when a user re-subscribes.

The idle threshold is configurable via $LJ::ESN_INACTIVE_DAYS in config-local.pl (documented in the example file). The same variable is used for the existing ProcessSub idle-skip check, replacing a previously hardcoded 365.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: 1c54be1b97b9d64d8833b984f942a9b66d326643 https://github.com/dreamwidth/dreamwidth/commit/1c54be1b97b9d64d8833b984f942a9b66d326643 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-11 (Sat, 11 Apr 2026)

Changed paths: M cgi-bin/DW/Task/ESN/FilterSubs.pm M cgi-bin/DW/Task/ESN/FindSubsByCluster.pm M src/dwtool/main.go

Log Message:

Fix false send failures and add CLI improvements

FilterSubs and FindSubsByCluster were passing the result of tasks_of_unique_matching_subs() directly to DW::TaskQueue->send(), which returns undef on an empty task list. When all subscriptions were legitimately filtered out by matches_filter, this was counted as a "failed send" (~47 false failures/min). Split the calls so empty task lists are a normal completion, not an error.

Also adds top-level help to dwtool listing available subcommands, and teaches esn-trace to accept a comment URL (parses ?thread= param, converts dtalkid to jtalkid, searches fired-event logs for the full trace ID automatically).

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: 034f12e28b75c1c3b3ce7b6ab173616484e33ee5 https://github.com/dreamwidth/dreamwidth/commit/034f12e28b75c1c3b3ce7b6ab173616484e33ee5 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-11 (Sat, 11 Apr 2026)

Changed paths: M cgi-bin/DW/Task/ESN/FilterSubs.pm M cgi-bin/DW/Task/ESN/FindSubsByCluster.pm M cgi-bin/DW/Task/ESN/FiredEvent.pm M cgi-bin/DW/Task/ESN/ProcessSub.pm M cgi-bin/LJ/ESN.pm M cgi-bin/LJ/Event/JournalNewComment.pm M cgi-bin/LJ/NotificationInbox.pm M cgi-bin/LJ/Subscription.pm M src/dwtool/main.go

Log Message:

Add deterministic ESN trace IDs and full pipeline observability

Every ESN log line now carries a deterministic trace prefix [esn ETYPEID:JOURNALID:ARG1:ARG2] derived from the event's raw_params, making end-to-end tracing possible by grepping a single string across all four ESN worker log groups. The trace ID is computable from a comment URL without needing to capture it at event-fire time.

Pipeline changes: - All four ESN task workers (FiredEvent, FindSubsByCluster, FilterSubs, ProcessSub) prefix every log line with the trace string - LJ::ESN::unique_matching_subs replaces the silent grep with a loop that logs each matches_filter rejection at DEBUG with user/sub/etypeid - JournalNewComment::matches_filter logs specific rejection reasons at each of its 9 return-0 paths via a $reject->() closure - Subscription::process logs notification construction failures and not_configured_for_user skips with trace prefix - NotificationInbox::enqueue logs successful delivery confirmation - FindSubsByCluster now checks TaskQueue->send() return values and logs users with >5000 subs that get silently skipped - FilterSubs now checks TaskQueue->send() return value - ProcessSub dead code fixed: subscription-not-found is correctly handled as a skip (COMPLETED) not routed through $failed (FAILED)

Metrics consolidation (Prometheus best practice): - All DW::Stats calls use one metric name per stage with a result tag instead of encoding the result in the metric name - dw.esn.firedevent result:started|completed|failed - dw.esn.findsubsbycluster result:started|completed|failed - dw.esn.filtersubs result:completed|dropped|failed - dw.esn.filter result:rejected - dw.esn.matches_filter result:rejected (with reason tag) - dw.esn.processsub result:processed|skipped|failed - dw.esn.process result:failed|skipped - dw.esn.inbox result:delivered|evicted

CLI tooling: - dwtool esn-trace searches all four ESN log groups in CloudWatch and pretty-prints a cross-stage timeline

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: 388d47d72bfa348ef6378e9f5e638e9fa85da491 https://github.com/dreamwidth/dreamwidth/commit/388d47d72bfa348ef6378e9f5e638e9fa85da491 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-11 (Sat, 11 Apr 2026)

Changed paths: M cgi-bin/DW/Task.pm M cgi-bin/DW/TaskQueue/LocalDisk.pm M cgi-bin/DW/TaskQueue/SQS.pm M etc/config-local.pl.example A t/taskqueue-serialize.t A t/taskqueue.t

Log Message:

Add JSON wire format for task queue with gradual rollout support

Adds serialize/deserialize methods to DW::Task that support a new v2 JSON wire format alongside the legacy Storable format. This enables non-Perl consumers (e.g. Go workers) to read task queue messages.

$LJ::TASK_QUEUE_JSON controls the rollout as a float 0-1 (e.g. 0.01 = 1% of messages serialized as JSON). Default is 0 (all Storable). All consumers can read both formats regardless of this setting. If JSON encoding fails for a task (e.g. blessed objects in args), it falls back to Storable with a warning and stat tag identifying the task class.

Also adds task queue config documentation to config-local.pl.example and comprehensive test coverage for LocalDisk lifecycle, serialization round-trips, and queue_attributes.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/main Home: https://github.com/dreamwidth/dreamwidth Commit: 727c496a47ff8257b62e438d302f0ca999c54d36 https://github.com/dreamwidth/dreamwidth/commit/727c496a47ff8257b62e438d302f0ca999c54d36 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-11 (Sat, 11 Apr 2026)

Changed paths: M src/dwtool/internal/aws/cloudwatch.go M src/dwtool/main.go A src/dwtool/run.sh

Log Message:

Add logscan CLI subcommand and build helper script to dwtool

Adds dwtool logscan for searching CloudWatch logs across all Dreamwidth services by keyword, with time range and log group glob filtering. Searches in 1-hour chunks (newest first) so results stream incrementally. Also adds run.sh to check dependencies, build, and run dwtool in one step.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

Commit: d81f640239516e9c21e95a6afdc73f4fc4d63156 https://github.com/dreamwidth/dreamwidth/commit/d81f640239516e9c21e95a6afdc73f4fc4d63156 Author: Mark Smith mark@dreamwidth.org Date: 2026-04-11 (Sat, 11 Apr 2026)

Changed paths: M cgi-bin/Plack/Middleware/DW/WriteTimeout.pm M t/plack-write-timeout.t

Log Message:

Tidy fixes

Compare: https://github.com/dreamwidth/dreamwidth/compare/fe8199f3512e...d81f64023951

To unsubscribe from these emails, change your notification settings at https://github.com/dreamwidth/dreamwidth/settings/notifications

Branch: refs/heads/dependabot/go_modules/src/dwtool/github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs-1.65.0 Home: https://github.com/dreamwidth/dreamwidth Commit: 0d0dde31d0f1e40afdb75a34408cfff9e453fd6c https://github.com/dreamwidth/dreamwidth/commit/0d0dde31d0f1e40afdb75a34408cfff9e453fd6c Author: dependabot[bot] <49699333+dependabot[bot]users> Date: 2026-04-08 (Wed, 08 Apr 2026)

Changed paths: M src/dwtool/go.mod M src/dwtool/go.sum

Log Message:

Bump github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs in /src/dwtool

Bumps github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs from 1.63.1 to 1.65.0. - Release notes - Commits