Future work

[stevenwdv]

• Make summary of existing JSON file?
• Check leaks on existing JSON file?
• Allow specifying all CLI options via a config file, clean up CLI options
• Add config & version to .crawl-state
• Better break up & test main.ts
• Log finished URLs and make it possible to continue stopped batch crawl
• Source maps for stack traces of request leaks & sniffs
• Option to simulate show password add-on (change type to text)
• Screen recordings (CDP Page.startScreencast)
• Detect password logged to console
• Option to update block lists
• Recursive link clicking
• Automatically detect filled values in headed mode, removing the need to fill the exact provided values
• Click field again after filling? Some trackers collect then
• Crawler won't fill fields if final URL has a different domain than initial URL; base check on final domain instead?
• Bias to filling form with password field, to avoid filling just newsletter form
• Max retries for URL in case of interrupt (out-of-memory error)
• Detect PII in URL (GET form)
• Split and clean up FieldsCollector code
• Generate HTML leak report
• Also capture all initial web requests, if possible