### What problem does this PR solve?
Improve concurrency in the knowledge compilation pipeline:
- Run Compile LLM requests concurrently while preserving ordered
commits.
- Run merge flush tasks concurrently while keeping ES writes ordered.
- Improve concurrency for local deduplication, chain validation, and ES
deduplication.
- Remove temporary debugging instrumentation and unused timing
variables.
### Type of change
- [x] Refactor (no functional change)
