Add --force option to reprocess items ignoring processed state#76
Merged
Add --force option to reprocess items ignoring processed state#76
Conversation
When processing code changes, previously-processed submissions, facts, and forms need to be re-parsed and re-stored. The --force flag bypasses the incremental processing checks in update, sync, and bootstrap commands, causing all items to be reprocessed. https://claude.ai/code/session_0142e35M3KT37gWcbj4LnGSW
Interfaces don't have an implicit index signature, so they fail the DataPorts constraint required by the Task generic. Type aliases work because they're structurally compatible with index signatures. https://claude.ai/code/session_0142e35M3KT37gWcbj4LnGSW
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a --force flag to the SEC pipeline CLI to bypass “already processed” incremental checks, enabling full reprocessing (re-parse/re-store) of submissions, company facts, and forms across update, sync, and bootstrap flows.
Changes:
- Add
force?: booleanto update/bootstrap task inputs and bypass processed-state filtering when enabled. - Wire
--forcethroughupdate,sync, andbootstrapCLI commands into the relevant tasks. - Update forms processing to optionally select all filings for requested form types when forced.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| src/task/submissions/UpdateAllSubmissionsTask.ts | Adds force input to reprocess all CIKs regardless of processed submissions state. |
| src/task/submissions/BootstrapSubmissionsTask.ts | Adds force input to process all downloaded submission CIK files regardless of processed state. |
| src/task/forms/UpdateAllFormsTask.ts | Adds force input to process all filings for selected forms rather than only unprocessed ones. |
| src/task/facts/UpdateAllCompanyFactsTask.ts | Adds force input to reprocess all CIKs regardless of processed facts state. |
| src/task/facts/BootstrapCompanyFactsTask.ts | Adds force input to process all downloaded company facts CIK files regardless of processed state. |
| src/cli/groups/update.ts | Adds --force option and passes it into update tasks. |
| src/cli/groups/sync.ts | Adds --force option and passes it into submissions/facts/forms update tasks. |
| src/cli/groups/bootstrap.ts | Adds --force option (bootstrap + bootstrap ingest) and passes it into bootstrap/update tasks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When processing code changes, previously-processed submissions, facts,
and forms need to be re-parsed and re-stored. The --force flag bypasses
the incremental processing checks in update, sync, and bootstrap
commands, causing all items to be reprocessed.
https://claude.ai/code/session_0142e35M3KT37gWcbj4LnGSW