Skip to content

[ShadowTrackr] Refactor connector with configuration and API integration #5294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jabesq wants to merge 9 commits into
base: feat/5170-Shadowtrackr-verify
Choose a base branch
from
Open

[ShadowTrackr] Refactor connector with configuration and API integration #5294

jabesq wants to merge 9 commits into from

Conversation

@jabesq
Copy link
Member

@jabesq jabesq commented Nov 28, 2025 •
edited
Loading

Proposed changes

Changes:

  • Added new ShadowTrackr connector with capabilities for enriching observables.
  • Introduced configuration settings for API integration, including base URL and API key.
  • Implemented core functionality for processing IP addresses and adjusting scores based on false positive estimates.
  • Created necessary classes and methods for STIX object conversion and relationship management.
  • Updated Docker and sample configuration files to reflect new connector settings.

Related issues

Checklist

  • I consider the submitted work as finished
  • I have signed my commits using GPG key.
  • I tested the code for its functionality using different use cases
  • I added/update the relevant documentation (either on github or on notion)
  • Where necessary I refactored code to improve the overall quality

Further comments

@github-actions github-actions bot added the filigran team use to identify PR from the Filigran team label Nov 28, 2025
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-refactor branch from 4a99b08 to d21917a Compare December 1, 2025 14:32
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-verify branch from 6365d58 to 23baf0d Compare December 1, 2025 14:32
@jabesq jabesq marked this pull request as ready for review December 1, 2025 14:43
@jabesq
Copy link
Member Author

jabesq commented Dec 1, 2025 •
edited
Loading

Check Status Notes
Use self.helper.connector_logger.() instead of deprecated self.helper.log_() ✅ Pass  
Remove CONFIDENCE_LEVEL ✅ Pass  
Remove UPDATE_EXISTING_DATA ✅ Pass  
Proper use of traceback.print_exc() in main ✅ Pass  
no bare 'try: ... except: ...' ✅ Pass  
Verify coherence of README.md ✅ Pass Update to reflect new defaults for connector configurations.
Ensure requirements.txt lists all sub-dependencies explicitly ✖️ Improvement stix2 is missing and is currently imported via pycti sub-dependencies.
Review configuration files (config.yml.sample, docker-compose.yml, .env) ✅ Pass  
Validate error handling (avoid bare try/except) ✅ Pass  
Remove x_opencti_report_status ✅ Pass  
Confirm proper use of TLP values ✅ Pass  
Verify author metadata ✅ Pass  
Pydantic used to build and validate setting ✅ Pass  
Define CONNECTOR_TYPE directly in the application ✅ Pass  
connector is composer supported ✅ Pass  Verification done by running generate_connector_config_json_schema.sh locally
Ensure helper.listen is used ✅ Pass  
Ensure if not in scope → return original bundle ✅ Pass  
No use of variable helper.api… ?? Still use of helper.api to create labels with color
Config file samples ✅ Pass .env.sample is missing
Dockerfile ✅ Pass 

@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-refactor branch from fb04181 to fdaed6e Compare December 1, 2025 16:27
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-verify branch 2 times, most recently from 046ffcc to 00e2eb5 Compare December 4, 2025 15:48
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-refactor branch from fdaed6e to 623930a Compare December 4, 2025 15:48
@jabesq jabesq linked an issue Dec 5, 2025 that may be closed by this pull request
[Shadowtrackr] Verify + add connector in the catalog #5170
Open
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-refactor branch from 623930a to 8607336 Compare December 5, 2025 11:10
throuxel
throuxel reviewed Dec 5, 2025
View reviewed changes
internal-enrichment/shadowtrackr/src/connector/connector.py
if marking_definition["definition_type"] == "TLP":
tlp = marking_definition["definition"]

valid_max_tlp = self.helper.check_max_tlp(tlp, self.max_tlp)
Copy link
Member

@throuxel throuxel Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if len(opencti_entity["objectMarking"]) = 0, tlp is not defined and this line leads to an error.

Copy link
Member Author

@jabesq jabesq Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comes, as is, from the template (

connectors/templates/internal-enrichment/src/connector/connector.py

Line 120 in 8f26452

if len(opencti_entity["objectMarking"]) != 0:
), to be checked with someone with more seniority (@Powlinett, @Kakudou, @helene-nguyen)

throuxel
throuxel reviewed Dec 5, 2025
View reviewed changes
internal-enrichment/shadowtrackr/src/connector/connector.py Outdated

text = self._get_ip_info_msg(ip_info, score_lowered, date_shortened)
if text:
description += f"\n[ShadowTrackr] {text}"
Copy link
Member

@throuxel throuxel Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got a case where description was None and this leads to an error.

Suggested change
description += f"\n[ShadowTrackr] {text}"
description = (description or "") + f"\n[ShadowTrackr] {text}"

Copy link
Member Author

@jabesq jabesq Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, but let's keep using f-string fully ;) :

description = f"{description or ''}\n[ShadowTrackr] {text}"

@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-refactor branch from 8607336 to 7850b64 Compare December 5, 2025 13:46
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-verify branch from 4a57c86 to db83500 Compare December 5, 2025 13:46
throuxel
throuxel reviewed Dec 5, 2025
View reviewed changes
internal-enrichment/shadowtrackr/src/connector/connector.py
score_lowered = False
for threshold, decrement in SCORE_STEPS:
if false_positive_estimate > threshold:
score -= decrement
Copy link
Member

@throuxel throuxel Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got a case where score was None and this leads to an error.
You can check for score and return if empty.

jabesq added 6 commits December 8, 2025 15:45
@jabesq
feat: Implement ShadowTrackr connector with configuration and API int…
e34e76e 
…egration

- Added new ShadowTrackr connector with capabilities for enriching observables.
- Introduced configuration settings for API integration, including base URL and API key.
- Implemented core functionality for processing IP addresses and adjusting scores based on false positive estimates.
- Created necessary classes and methods for STIX object conversion and relationship management.
- Updated Docker and sample configuration files to reflect new connector settings.
@jabesq
chore: Update boilerplates and README
eb54a28
@jabesq
chore: Delete old shadowtrackr connector
24470b9
@jabesq
chore: Add missing .env.sample file
9385eb5
@jabesq
fix: Revert change of setting max_tlp into max_tlp_level
f92eece 
This change could be breaking with existing setup. Better to keep
setting name as it is.
@jabesq
fix: Handle case description is missing
9d81fff
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-refactor branch from 41f3c38 to 80e1986 Compare December 8, 2025 14:45
@jabesq jabesq force-pushed the feat/5170-Shadowtrackr-verify branch from db83500 to 27a4744 Compare December 8, 2025 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@throuxel throuxel throuxel left review comments

@Kakudou Kakudou Awaiting requested review from Kakudou

@helene-nguyen helene-nguyen Awaiting requested review from helene-nguyen

@mariot mariot Awaiting requested review from mariot

@Megafredo Megafredo Awaiting requested review from Megafredo

@Ninoxe Ninoxe Awaiting requested review from Ninoxe

@Powlinett Powlinett Awaiting requested review from Powlinett

Assignees

@jabesq jabesq

Labels

filigran team use to identify PR from the Filigran team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Shadowtrackr] Verify + add connector in the catalog

3 participants

@jabesq @throuxel