The evaluation reveals when heterogeneity-awareness matters most:

1. High cluster load (queues build up, smart allocation prevents explosion)
2. Diverse workloads (mix of jobs with different GPU preferences)
3. Diverse hardware (more GPU types = more optimization opportunity)

If your cluster has one GPU type or one job type, Gavel reduces to the baseline. The win comes from matching diverse jobs to diverse hardware.

The Shockwave sample shows the typical CS244 scope: 2 students reproduced one key figure from the paper by implementing ~3 core components. For 4 people, we need roughly double this scope with clearly separable work streams.

Internal hyperlinks in python-docx: Word documents use two mechanisms for internal links: (1) bookmarks - invisible anchors at target locations, and (2) hyperlinks with w:anchor attribute pointing to bookmark names. Since python-docx doesn’t expose these directly, we manipulate the underlying OOXML elements via OxmlElement and the qn() namespace helper.

Word internal links: Unlike HTML anchors, Word uses a two-part system: <w:bookmarkStart> marks the destination with a unique ID and name, then <w:hyperlink w:anchor="name"> links to it. The ID must be unique across all bookmarks in the document, while the name is what hyperlinks reference.

Section-scoped state: The pattern used here - a global _cited_in_section set that gets cleared on section boundaries - is a lightweight way to implement “first occurrence per scope” logic without restructuring the entire parser. It trades some purity for simplicity compared to passing state through every function.

SQL Server CASCADE Constraint Limitation: SQL Server prevents multiple CASCADE paths to the same table because it can’t determine which path to follow during deletion. Here, deliveries already cascades to users via subscriptions (subscription → user), so adding a direct deliveries.user_id → users.user_id CASCADE creates ambiguity. The fix is to use NO ACTION or SET NULL instead.

Summary of the fix:

1. Root Cause: SQL Server rejects multiple CASCADE paths to the same table. The deliveries table already cascades to users via subscriptions, so a direct user_id → users CASCADE created a cycle.
2. Fix: Changed ondelete="CASCADE" to ondelete="NO ACTION" for the user_id FK, matching the existing pattern used for validation_job_id.
3. Impact: When a user is deleted, direct deliveries with that user_id will not be automatically deleted (you’d need to handle this in application code or via a separate cleanup process).

Situation: The code fix for NULLS LAST is correct (unit tests pass), but the GPU Docker image build is failing because Ubuntu’s Launchpad PPA service is experiencing an outage. This prevents the workflow from completing.

Options:

1. Wait for Launchpad to recover and re-run
2. Merge with admin bypass since this is infrastructure-related, not code-related
3. The fix is in PR #93 - you can merge when ready

CLAUDE.md Design Philosophy: The purpose of CLAUDE.md is to give AI assistants (and developers) the context they need to work effectively without reading every design doc. Key principles:

1. Constraints first - What NOT to do is often more valuable than what to do
2. Decisions over descriptions - Capture the “why” behind architectural choices
3. Single source of truth - Reference detailed docs rather than duplicating

Key improvements based on best practices:

1. Progressive disclosure - References design docs instead of duplicating content
2. Constraints-first - DO NOT VIOLATE table is prominent and expanded
3. Removed redundancy - Dropped env vars table (duplicates config.py), testing strategy (covered in Commands)

Current summarization flow:

• CPU processor claims deliveries with ready transcripts (line 455-479 in cpu_processor.py)
• Calls LLMProvider.generate_summary() synchronously for each transcript
• Has a TokenRateLimiter that tracks TPM/RPM and waits when approaching Azure quotas
• One-at-a-time processing with 50k TPM limit

Why batch API fits your philosophy: The CLAUDE.md states “prefer batching and scale-to-zero over low latency. Optimize for cost, not speed” - this is exactly what the batch API is designed for (50% cost savings, 24-hour SLA).

Azure OpenAI Batch API basics:

• Requires a Global Batch deployment (separate from your standard gpt-5.2-chat deployment)
• You upload a JSONL file to blob storage with all requests
• Azure processes asynchronously, writes results back to blob
• 50% cost savings vs standard pricing
• 24-hour SLA (typically faster in practice)
• Results available for 24 hours after completion

Good news: You already have Azure Blob storage set up, which is required for batch input/output files.

The daily limit SQL is dead code. The sophisticated claim_deliveries function with daily limit logic exists in cpu_processor.py:284, but the actual production path (run_cpu_processor -> delivery_service.deliver_pending) never calls it.

• Tests pass because they call claim_deliveries directly
• Production fails because deliver_pending uses its own simple query with no daily limit

This is a classic integration gap - the feature was implemented and unit tested, but never wired into the actual code path.

The Bug: The claim_deliveries function with daily limit logic was implemented but never called by the production code path. The run_cpu_processor function called delivery_service.deliver_pending() which had its own simple query with no daily limit checking.

Why tests passed: The existing tests called claim_deliveries directly, not through run_cpu_processor. Classic integration gap.

The Fix: Three changes:

1. Modified run_cpu_processor to use claim_deliveries() instead of bypassing it
2. Added pre_claimed parameter to deliver_one() to skip re-claiming
3. Updated status methods (_mark_sent, _mark_failed, _mark_retry, _release_delivery) to clear processing_owner_id

Why heredocs fail here:

1. do shell script in AppleScript passes commands to /bin/sh -c as a single string
2. Heredocs (<< 'EOF') require multi-line handling that doesn’t work well in this context
3. HTML contains many shell metacharacters (<, >, $, backticks, quotes) that cause parsing errors

Why native AppleScript file I/O is better here:

1. No shell escaping needed - writes raw bytes directly
2. «class utf8» ensures proper UTF-8 encoding for international characters
3. set eof of fileRef to 0 clears any existing content before writing
4. The nested try in the error handler ensures the file handle is always closed

Key AppleScript improvements I’ll make:

1. Native file I/O - Bypasses shell entirely, avoiding all escaping issues
2. Better PDF detection - Handle URLs with query strings (e.g., file.pdf?token=xxx)
3. curl timeout - Prevent hanging on slow/unresponsive servers
4. Cleaner error handling - Consolidated cleanup in error paths
5. Remove scroll-to-bottom - This was a visual indicator but feels unnecessary

Why native AppleScript I/O is robust:

• «class utf8» ensures proper encoding for any Unicode content
• set eof of fileRef to 0 truncates any existing content before writing
• The nested try in error handling guarantees the file handle is always closed, preventing resource leaks
• No shell interpolation means <script>, $variables, and backticks are written literally

Why trafilatura was chosen:

• It’s a battle-tested library for extracting article content from messy HTML
• Handles boilerplate removal, ads, nav elements, etc.
• But it has ~15 transitive dependencies (lxml, urllib3, certifi, etc.)

How the embedded JS converter works:

1. Recursively walks the DOM tree, converting each element to Markdown
2. Handles all common elements: headers, paragraphs, lists, links, images, code blocks, tables, blockquotes
3. Leverages Safari’s Reader Mode when available for pre-cleaned content
4. Falls back to finding <article>, <main>, or similar semantic elements

How image downloading works in the JS version:

1. Extract - grep parses markdown for ![alt](url) patterns
2. Resolve - Handles relative URLs (/path, //host/path, path)
3. Download - curl fetches with proper User-Agent and 15s timeout
4. Dedupe - Adds counter suffix if filename already exists
5. Rewrite - sed replaces markdown syntax with ![[folder/file.jpg]]